72
METR AI Coding Research Inconclusive Because Dev Participants Refused to Complete Tasks Without AI (metr.org)
submitted 5 months ago* by to c/programming@programming.dev
23 comments fedilink hide all child comments
 

Selected developer quotes:

“I’m torn. I’d like to help provide updated data on this question but also I really like using AI!” — a developer from the original study early-2025 when asked to participate in the late-2025 study.

“I found I am actually heavily biased sampling the issues … I avoid issues like AI can finish things in just 2 hours, but I have to spend 20 hours. I will feel so painful if the task is decided as AI-disallowed.” — a developer from the new study noting selection effects when choosing what tasks to include in the study.

“my head’s going to explode if I try to do too much the old fashioned way because it’s like trying to get across the city walking when all of a sudden I was more used to taking an Uber.” — a developer from the new study noting selection effects when choosing what tasks to include in the study.

you are viewing a single comment's thread
view the rest of the comments
[–] 1 point 4 months ago*

Yeah I know, but if you really mean that analogy then the conclusion is that the normal thing for 99% of programmers to do should be to use AI. In the same way that 99% of people do not get around by running.

I don't agree with that yet - so far I've found AI to be a very fast but mediocre programmer. Kind of like giving a beginner access to all the documentation and a time machine. Sometimes that's exactly what you want. But definitely not most of the time.

  • permalink
  • fedilink
  • source
  • parent