top 50 comments

sorted by: hot top controversial new old
[+] 288 points 1 year ago* (last edited 1 year ago) (20 children)
  • [–] 230 points 1 year ago (15 children)

    AI including ChatGPT is being marketed as super awesome at everything, which is why that and similar AI is being forced into absolutely everything and being sold as a replacement for people.

    Something marketed as AGI should be treated as AGI when proving it isn't AGI.

  • source
  • parent
  • hideshow 16 child comments
  • [–] 14 points 1 year ago (12 children)

    Not to help the AI companies, but why don't they program them to look up math programs and outsource chess to other programs when they're asked for that stuff? It's obvious they're shit at it, why do they answer anyway? It's because they're programmed by know-it-all programmers, isn't it.

  • source
  • parent
  • hideshow 14 child comments
  • [–] 27 points 1 year ago

    why don't they program them

    AI models aren't programmed traditionally. They're generated by machine learning. Essentially the model is given test prompts and then given a rating on its answer. The model's calculations will be adjusted so that its answer to the test prompt will be closer to the expected answer. You repeat this a few billion times with a few billion prompts and you will have generated a model that scores very high on all test prompts.

    Then someone asks it how many R's are in strawberry and it gets the wrong answer. The only way to fix this is to add that as a test prompt and redo the machine learning process which takes an enormous amount of time and computational power each time it's done, only for people to once again quickly find some kind of prompt it doesn't answer well.

    There are already AI models that play chess incredibly well. Using machine learning to solve a complexe problem isn't the issue. It's trying to get one model to be good at absolutely everything.

  • source
  • parent
  • load more comments (10 replies)
  • load more comments (14 replies)
  • [–] 10 points 1 year ago (3 children)

    I agree with your general statement, but in theory since all ChatGPT does is regurgitate information back and a lot of chess is memorization of historical games and types, it might actually perform well. No, it can't think, but it can remember everything so at some point that might tip the results in it's favor.

  • source
  • parent
  • hideshow 3 child comments
  • load more comments (3 replies)
  • load more comments (15 replies)
    [–] 84 points 1 year ago (5 children)

    Tbf, the article should probably mention the fact that machine learning programs designed to play chess blow everything else out of the water.

  • source
  • hideshow 8 child comments
  • [–] 15 points 1 year ago

    I forgot which airline it is but one of the onboard games in the back of a headrest TV was a game called “Beginners Chess” which was notoriously difficult to beat so it was tested against other chess engines and it ranked in like the top five most powerful chess engines ever

  • source
  • parent
  • [–] 13 points 1 year ago* (last edited 1 year ago)

    Machine learning has existed for many years, now. The issue is with these funding-hungry new companies taking their LLMs, repackaging them as "AI" and attributing every ML win ever to "AI".

    ML programs designed and trained specifically to identify tumors in medical imaging have become good diagnostic tools. But if you read in news that "AI helps cure cancer", it makes it sound like it was a lone researcher who spent a few minutes engineering the right prompt for Copilot.

    Yes a specifically-designed and finely tuned ML program can now beat the best human chess player, but calling it "AI" and bundling it together with the latest Gemini or Claude iteration's "reasoning capabilities" is intentionally misleading. That's why articles like this one are needed. ML is a useful tool but far from the "super-human general intelligence" that is meant to replace half of human workers by the power of wishful prompting

  • source
  • parent
  • load more comments (2 replies)
    [–] 76 points 1 year ago* (last edited 1 year ago) (5 children)

    An LLM is a poor computational/predictive paradigm for playing chess.

  • source
  • hideshow 7 child comments
  • load more comments (3 replies)
    [–] 61 points 1 year ago (12 children)

    ChatGPT has been, hands down, the worst AI coding assistant I've ever used.

    It regularly suggests code that doesn't compile or isn't even for the language.

    It generally suggests AC of code that is just a copy of the lines I just wrote.

    Sometimes it likes to suggest setting the same property like 5 times.

    It is absolute garbage and I do not recommend it to anyone.

  • source
  • hideshow 14 child comments
  • [–] 17 points 1 year ago (4 children)

    I find it really hit and miss. Easy, standard operations are fine but if you have an issue with code you wrote and ask it to fix it, you can forget it

  • source
  • parent
  • hideshow 5 child comments
  • load more comments (3 replies)
  • load more comments (10 replies)
    [–] 50 points 1 year ago (5 children)

    Sometimes it seems like most of these AI articles are written by AIs with bad prompts.

    Human journalists would hopefully do a little research. A quick search would reveal that researches have been publishing about this for over a year so there's no need to sensationalize it. Perhaps the human journalist could have spent a little time talking about why LLMs are bad at chess and how researchers are approaching the problem.

    LLMs on the other hand, are very good at producing clickbait articles with low information content.

  • source
  • hideshow 6 child comments
  • [–] 24 points 1 year ago (7 children)

    Gotham chess has a video of making chatgpt play chess against stockfish. Spoiler: chatgpt does not do well. It plays okay for a few moves but then the moment it gets in trouble it straight up cheats. Telling it to follow the rules of chess doesn't help.

    This sort of gets to the heart of LLM-based "AI". That one example to me really shows that there's no actual reasoning happening inside. It's producing answers that statistically look like answers that might be given based on that input.

    For some things it even works. But calling this intelligence is dubious at best.

  • source
  • parent
  • hideshow 7 child comments
  • load more comments (7 replies)
  • load more comments (4 replies)
    [–] 45 points 1 year ago* (5 children)

    I suppose it's an interesting experiment, but it's not that surprising that a word prediction machine can't play chess.

  • source
  • hideshow 5 child comments
  • load more comments (5 replies)
    [–] 43 points 1 year ago (3 children)

    I swear every single article critical of current LLMs is like, "The square got BLASTED by the triangle shape when it completely FAILED to go through the triangle shaped hole."

  • source
  • hideshow 4 child comments
  • [–] 42 points 1 year ago (4 children)

    It's newsworthy when the sellers of squares are saying that nobody will ever need a triangle again, and the shape-sector of the stock market is hysterically pumping money into companies that make or use squares.

  • source
  • parent
  • hideshow 5 child comments
  • load more comments (3 replies)
  • load more comments (2 replies)
    [–] 41 points 1 year ago (1 child)

    LLM are not built for logic.

  • source
  • hideshow 2 child comments
  • [–] 18 points 1 year ago (2 children)

    And yet everybody is selling to write code.

    The last time I checked, coding was requiring logic.

  • source
  • parent
  • hideshow 3 child comments
  • [–] 10 points 1 year ago (4 children)

    To be fair, a decent chunk of coding is stupid boilerplate/minutia that varies environment to environment, language to language, library to library.

    So LLM can do some code completion, filling out a bunch of boilerplate that is blatantly obvious, generating the redundant text mandated by certain patterns, and keeping straight details between languages like "does this language want join as a method on a list with a string argument, or vice versa?"

    Problem is this can be sometimes more annoying than it's worth, as miscompletions are annoying.

  • source
  • parent
  • hideshow 4 child comments
  • load more comments (4 replies)
  • load more comments (1 reply)
  • [–] 36 points 1 year ago (6 children)

    A strange game. How about a nice game of Global Thermonuclear War?

  • source
  • hideshow 7 child comments
  • load more comments (5 replies)
    [–] 29 points 1 year ago (6 children)

    Can ChatGPT actually play chess now? Last I checked, it couldn't remember more than 5 moves of history so it wouldn't be able to see the true board state and would make illegal moves, take it's own pieces, materialize pieces out of thin air, etc.

  • source
  • hideshow 6 child comments
  • load more comments (6 replies)
    [–] 25 points 1 year ago

    Ah, you used logic. That's the issue. They don't do that.

  • source
  • [–] 20 points 1 year ago (3 children)

    Hardly surprising. Llms aren't -thinking- they're just shitting out the next token for any given input of tokens.

  • source
  • hideshow 3 child comments
  • load more comments (3 replies)
    [+] 15 points 1 year ago* (last edited 1 year ago) (2 children)
  • load more comments (1 reply)
    [–] 15 points 1 year ago

    All these comments asking "why don't they just have chatgpt go and look up the correct answer".

    That's not how it works, you buffoons, it trains off of datasets long before it releases. It doesn't think. It doesn't learn after release, it won't remember things you try to teach it.

    Really lowering my faith in humanity when even the AI skeptics don't understand that it generates statistical representations of an answer based on answers given in the past.

  • source
  • [–] 13 points 1 year ago (4 children)

    Using an LLM as a chess engine is like using a power tool as a table leg. Pretty funny honestly, but it's obviously not going to be good at it, at least not without scaffolding.

  • source
  • hideshow 4 child comments
  • load more comments (4 replies)

    I mean, that 2600 Chess was built from the ground up to play a good game of chess with variable difficulty levels. I bet there's days or games when Fischer couldn't have beaten it. Just because a thing is old and less capable than the modern world does not mean it's bad.

  • source
  • [–] 11 points 1 year ago (3 children)

    I'm often impressed at how good chatGPT is at generating text, but I'll admit it's hilariously terrible at chess. It loves to manifest pieces out of thin air, or make absurd illegal moves, like jumping its king halfway across the board and claiming checkmate

  • source
  • hideshow 3 child comments
  • load more comments (3 replies)
    [–] 10 points 1 year ago

    Can i fistfight ChatGPT next? I bet I could kick its ass, too :p

  • source
  • [–] 9 points 1 year ago (4 children)

    There was a chess game for the Atari 2600? :O

    I wanna see them W I D E pieces.

  • source
  • hideshow 5 child comments
  • load more comments (3 replies)
    load more comments
    view more: next ›