Omg the comments are so out of hand. I regularly do code reviews on colleagues who use AI to write code (some whilst protesting, but still). The comments are usually the worst part.

The thing writes entire novels in the summary that do nothing but confuse and add cognitive load. It adds comments to super obvious things, describing what the code does instead of why. Yes AI I can read code, I know assigning a variable a value is how shit works. And I have still got PTSD from those kinds of comments from a legacy system I worked on for years that did the exact same, except the comments and the code didn't match up, so it was a continuous guess which one was the intended one. It also likes to put responses to the prompt in the comments. So for example when it assigned A to a variable and it was supposed to be B, when you point this out it adds a comment saying something like: This is supposed to be B not A. But when you read those comments after the fact, it makes zero sense. Like of course it should be B? Why should it ever be A?

And it often generates a bunch of markdown docs which are plain drivel, luckily most devs just delete those before I see them.

My personal experience is in 30% of cases the AI is just plain wrong and the result is nonsense, delete that shit and try again. In the 70% that does have some kind of answer there is ALWAYS at least one big issue and usually multiple. It's a 50/50 if the code is workable with some kinks to work out, or if it is seriously flawed and needs a lot of work. For experienced devs it can be a helpful thing if they have writers block, to give them something to be angry about, showing them how they can do better. But for inexperienced devs it's just plain terrible, the code is shit and the dev doesn't even know. And worse still the dev doesn't learn. I try to sit down with them, explain the shortcomings and how to do better. But they don't learn, they just know what stuff to write in the prompt, in order to not get me on their case. Or they will say stuff like: but it works right? Facepalm

That company I do work for also tried getting their sysadmins and devops people to use AI. Till one day there was a permissions issue, which admittedly was pretty complicated, where they ended up solving it with AI. The team was happy, the upper management was happy, high fives all around. Till the grumpy old sysadmin who has 40 years of experience takes a look and hits the big ol' red alarm button of doom. Full investigation later, the AI had fucked up and created a huge hole in the security. There was zero evidence it had been exploited, but that doesn't matter. All the work still needed to be done, all the paperwork filed, proper agencies informed, because the security issue was there. Management eased up on AI usage for those people real fast.

It's so weird how people in charge want to use AI, but aren't even really sure of what it is and what it isn't. And they don't listen to what the people with actual knowledge have to say. In their minds we are probably all just covering our asses to not be out of a job.

But for real if anyone in management is listening, take it from an old asshole who has done this job since the 80s: AI fucking sucks!

AI coding tools definitely helpful with boilerplate code

They're really not. Just because they generated a starter template for you doesn't mean you actually needed all of that mountain of slop. My coworker recently did a presentation where he generated a starter project for a Go project and most of it was shit and just not necessary. People assume you need mountains of boilerplate, but you may not need that. (Worse, AI is cementing bad practices at work.)

But also, assuming your project does need to generate a ton of boilerplate, should you really be going to the casino and rolling for a fresh mountain of slop that is hopefully correct? We can already generate code: snippets (in your editor), templates (like cloning a template repo), and generators (like create-react-app) already exist. Aaand these are deterministic, debuggable, and fixable.

I am interested to see if these tools can be used to tackle tech debt

Having it rewrite existing functioning code seems like a terrible idea. QA would at least have to re-test all functionality.

These might be of interest to software developers but it's all just style nothing here actually effects the computation. The problem I encounter with LLMs is that they are incapable of doing anything but rehearsing the same algorithms you get off of blogs. I can't even successfully force them to implement a novel algorithm they will simply deny that it is valid and revert back to citing their training data.

I don't see LLMs actually furthering the field in any real way ( even if by accident, since they can't actually perform deductive reasoning).

I am interested to see if these tools can be used to tackle tech debt, as often the argument for not addressing tech debt is a lack of time, or if they would just contribute it to it, even with thorough instructions and guardrails.

From my experience working with people who use them heavily, they introduce new ways of accumulating tech debt. Those projects usually end up having essays of feature spec docs, prompts, state files (all in prose of course), etc. Those files are anywhere from hundreds to thousands of lines long, and there's a lot of them. There's no way anybody is spending hours reading through enough markdown to fill twenty encyclopedia-sized books just to make sure it's all up-to-date. At least, I can promise that I won't be doing it, nor will anyone I know (including those using AI this way).

From what I've seen I wouldn't trust it to tackle technical debt. Quite the opposite, I'd use LLM ro build and MVP, then consider it's output to be technical debt that will have to be cleaned up over time as the product matures,.by someone who knows what he's doing.

LLM right now is more of a junior developer.