This is the same reason I had to turn off my search engines crawler.

There were changes made to the API to ignore any page > 99. So if you ask for page 100 or page 1_000_000_000 you get the first page again. This would cause my crawler to never end in fetching "new" posts.

lemm.ee on the other hand made a similar change but anything over 99 returns an empty response. lemm.ee also flat out ignores sort=Old, always returning an empty array.

Both of these servers did it for I assume the same reason. Using a high page number significantly increases the response time. It used to be (before they blocked pages over 99) that responses could take over 8-10 seconds! But asking for a low page number would return in 300ms or less. So because it's a lot harder to optimize the existing queries, and maybe not possible, for now the problematic APIs were just disabled.

Oh wow, I thought this was a bug in the Lemmy api. I was implementing comments paging in my app the other day and noticed it would just infinity load pages with duplicate comments. I guess I should have tested with another instance before disabling the feature.

Can you check periodically so you can get the link?