173
How anyone thinks they are productive with llm tools is baffling (sh.itjust.works)
submitted 4 months ago* by to c/fuck_ai@lemmy.world
96 comments fedilink hide all child comments
 

It has to be pure ignorance.

I only have used my works stupid llm tool a few times (hey, I have to give it a chance and actually try it before I form opinions)

Holy shit it's bad. Every single time I use it I waste hours. Even simple tasks, it gets details wrong. I correct it constantly. Then I come back a couple months later, open the same module to do the same task, it gets it wrong again.

These aren't even tools. They're just shit. An idiot intern is better.

Its so angering people think this trash is good. Get ready for a lot of buildings and bridges to collapse because of young engineers trusting a slop machine to be accurate on details. We will look back on this as the worst era in computing.

you are viewing a single comment's thread
view the rest of the comments
[–] 2 points 4 months ago* (1 child)

Claude and super powers / planning have changed my mind more on AI feature development. Iterating on the spec and making it as unambiguous as possible gives good results when you clear context and have it implement the plan. Even if it starts to stray you can just do a git reset and start a new session with the spec, adjusting it a bit, because time wise you probably haven't invested much.

It also depends on the code base, if the code base has very clear separation of concerns, good documentation, and good contracts between layers then claude can handle it pretty well. If the code base is full of spaghetti code with multiple ways to do the same thing then AI will struggle with it. In our large legacy monolith repo it doesn't do well, in our micro service repos it does great.

Also time wise it may not seem like a benefit if you just set it and wait for it to complete, the productivity advantage comes from running a couple sessions in parallel.

Also context is key, having a good claude.md file in the repo to explain patterns helps it to avoid pitfalls. If it's only context is the prompt you gave it and you tell it to implement a feature without a plan / spec outlined it will generate shit code.

  • permalink
  • fedilink
  • source
  • hideshow 2 child comments
    •
  • [–] 13 points 4 months ago* (2 children)

    Making it as unambiguous as possible

    If only we had a way to communicate with machines in a reliable, deterministic and unambiguous way.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] -1 points 4 months ago (1 child)

    Yeah you can write the code yourself. You can also write in c or even assembly if you really want to make it as unambiguous as possible, it'll just take more time. Some people like to code in Python though because they can write faster with it even if a lot of implementation details and choices are hidden from them because they don't care about those details.

    Spec driven development in my view is just another step, albeit a big one, on the level of abstraction between assembly and python. Like python it has its places and has places where it should never be used for safety and performance reasons.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 4 points 4 months ago (1 child)

    They may not care about the implementation details of a Python library, they do care about consistent execution and predictable results. And in some edge cases, they will care about the documentation saying exactly how those edge cases are handled.

    Writing Python is abstraction, yes, but it's still programming. Once that Python code is written and tested and the dependencies are locked down, you can ship it and be certain it always works as designed.

    Spec-driven code generation is nothing like that. I can't ship the specs. I could generate the code in a pipeline and ship that, maybe. But there's no way I'm getting consistent builds from a code generator. So what do people do? They generate the code and put it in source control for review. When have you ever checked-in a compiled executable or looked at it? There's machine code in there, shouldn't you review that the compiler did what you asked of it?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] -1 points 4 months ago*

    Consistency is dependent on the code base and not the "compiler" in this sense. If the code base has consistent patterns and only has one well documented way to implement something then the AI will follow those patterns, ie. If there is only one way to run a job, AI will use that method. There might be some variation in variable names, formatting, etc. but the core flow should be consistent between "runs"

    You can and should still test your code , both manually and with automation to ensure it does what it says it does. Testing should be the way you are certain it always works as designed. IMO understanding your tests and test coverage is more important than understanding the implementation. This is why part of the spec for superpowers is a test plan, and that should be the most reviewed / iterated part.

  • permalink
  • fedilink
  • source
  • parent