441
Amazon Shuts Down Internal AI Leaderboard After Employees Cheated (www.404media.co)
submitted 2 months ago by to c/technology@lemmy.world
91 comments fedilink hide all child comments
 

Amazon has shut down an internal company leaderboard which ranked employees based on how much they used AI tools at work.

you are viewing a single comment's thread
view the rest of the comments
[–] 230 points 2 months ago (7 children)

Whole concept of "how much AI you used", is so flipping stupid of a metric I can't even wrap my head around. I mean even if we assume it's their own AI they are using... that's their power etc... That's like a leaderboard for most gas used up, or miles driven by your truck drivers.

  • permalink
  • fedilink
  • source
  • hideshow 14 child comments
    •
  • [–] 116 points 2 months ago (1 child)

    It's the old lines of code metric style of thinking.

    The people in charge only know line go up means better and bigger number is better.

    Its only when things go wrong and someone ELI5 for them that they listen. And even then it'll wear off in a week and they'll be on to their next make the number bigger obsession.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 27 points 2 months ago (3 children)

    I've had to explain this to more executives than I wish to remember. Computer code is a recipe, not a cake. When you see a recipe that's super long, and requires two kitchens worth of bakeware and tools, you probably think it's a bad recipe. Short, elegant, easy to follow recipes with a little note in the margin from your grandmother about what to do when the dough is too sticky are the best recipes.

    Unfortunately, one learned the exact wrong lesson from this, and started measuring lower lines of code produced as better... Which worked for a while, but lead to a lot of weirdness around new features for no particular reason.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 6 child comments
    •
  • [–] 5 points 2 months ago

    "Sorry, can't add that feature. That would take at least another 20 lines of code and I can't lose my leaderboard ranking. Not with Steven in second place..."

    I've said it for years: qualitative is better than quantitative. It's just that it doesn't look as nice and objective on reports. This obsession with numbers has been a bane on society for a long time.

  • permalink
  • fedilink
  • source
  • parent
  • [–] 4 points 2 months ago

    Unfortunately, one learned the exact wrong lesson from this, and started measuring lower lines of code produced as better... Which worked for a while, but lead to a lot of weirdness around new features for no particular reason.

    Hey boss, I just got that new feature submitted for review. I managed to get it all on a single line of code! No, that one line has like 5000 characters, why do you ask?

  • permalink
  • fedilink
  • source
  • parent
  • [–] 14 points 2 months ago (1 child)

    Right? How do you even cheat on a ruleset this dumb? You don‘t. It was a stupid contest from the start.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 11 points 2 months ago (1 child)

    If they're ranking you based on AI usage, they're going to use that when they make decisions about raises and promotions. It may be a stupid contest but the employees are part of it whether they like it or not.

    You can bet your ass I'd be trying to pad my numbers.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 4 points 2 months ago (1 child)

    If they're ranking you based on AI usage, they're going to use that when they make decisions about raises and promotions.

    By not promoting the people on top of the ranking who can't do things without AI and waste a lot of resources, right?

    ...right?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 11 points 2 months ago (1 child)

    It only makes sense if

    • you want to drive up adoption because
      • you're confident in usefulness already
      • want to find out about usefulness and need the userbase and usage for it
    • you have ulterior motives to push for AI adoption

    I can imagine leadership - disconnected from real work and any practical AI use experience - being misinformed and misguided into believing marketing and hype-cycle about gains. It also doesn't seem implausible that leadership wants to drive up adoption to quickly gain feedback and results about usefulness and gains/loss.

    In good faith, it requires a certain mindset (no care about the waste or potential loss or risk) and distance from practice. Not implausible, though, in my eyes.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 3 points 2 months ago (1 child)

    personally oversaw a 300% increase in lines of code committed. 40% reduction in delays and 60% reduction in feature implementation design cycles. As a result, increased company revenue by 30%

    This is all the explanation you need on why they're doing this bone headed shit. It's not their problem in a few quarters when they jump ship after padding their resume on the company's dime.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 2 months ago*

    Sure. But I haven't seen credible evidence that any of this drove any revenue.

    Correct features, thoughtfully planned, and expertly executed, at the right time, sometimes drive new revenue.

    AI slop is about 99% orthogonal to anything that helps drive revenue.

    People will claim it helps with timing, but timing only works if the feature is correct and AI makes organizations that rarely got things correct in the first place even less likely to get things correct.

  • permalink
  • fedilink
  • source
  • parent
  • [–] 6 points 2 months ago (1 child)

    Like judging how good a driver is by the person with the longest distance a to b.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 2 months ago (2 children)

    I mean I can say it's not quite as guaranteed to be bad. IE distance from A to B is inherently bad, Most distance driven COULD mean you made more stops and did more good.... or it could mean you took a horrible route, got lost and wound up crossing through the wrong state.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 1 point 2 months ago

    or it could mean you took a horrible route, got lost and wound up crossing through the wrong state.

    We don't need a map to guide us.

    Should it be snowing?

    Hey, I've never seen the sun come up in the West?!

  • permalink
  • fedilink
  • source
  • parent
  • [+] -7 points 2 months ago (2 children)

    I mean miles driven is at least proportional to amount of useful work, more miles is more distance they've transported cargo. The AI usage leaderboard is useless.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 17 points 2 months ago* (last edited 2 months ago) (1 child)

    Only if you assume that all miles are correctly headed towards the destination. 2 drivers are given the same 3 places to deliver packages to.

    one drives 2000 miles, the other 50.

    Admitted though now that I actually think about it, I guess we already have that level of stupid assumptions. After all the normal system of pay for many jobs is by the hour. Which I suppose has all the same flaws.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 2 months ago

    After all the normal system of pay for many jobs is by the hour. Which I suppose has all the same flaws.

    Yes! It does. I felt that I had to get out of hourly billing consultancy programming, because I code so fast I would have gone broke.

  • permalink
  • fedilink
  • source
  • parent