Matt Parker (stand up maths on YouTube) has a video from the last solved problem about how they did it.

I'm also way more interested in the details than the average news article on how AI can do maths now. Like how did they home in on this specific problem to solve? Destiny? Did they brute-force it and try 5000 other unsolved maths problems first? Was it AI assisted or does AI do this on its own? What's the prompt? How many tokens did it burn through? Has it been lucky by chance or would I be able to input the same thing and there's a 90% chance it'll solve it again?

Wikipedia already has a full write up. It looks like Fable simply found an example of a function that had a constant Jacobian determinant (the determinant of the matrix of partial derivatives), but the function was not injective (3 different values of x, y, and z give the function the same value). Basically, the function is not "smooth", so it can't be inverted.

I'm guessing that you'd...

1: Choose a math problem that's difficult to solve, but easy to verify whether a solution is correct or not.

2: Set up a loop of two AIs talking to each other (or just two instances of one AI). The first is set up to come up with as many different solutions for the problem as possible, repeatedly, for many thousands of attempts. The second is tasked with checking each attempt to see whether it is actually a solution or not.

3: When the second AI flags something as a valid solution, have a human look over and check to see whether it really is or whether the second AI is just hallucinating. Hopefully, among just a few dozen false positives, you'll find the actual valid solution.