That's my thought as well. And that's how I always go when entering a new hobby. Buy cheap that has great impressions, and judge from there.
After some sorting my mind, I think I would want to start just with ps1 and GBA. That is my childhood gaming systems after all. My parents didn't get any of the Nintendo systems prior to the ps1 as far as I can remember.
So since I've always wanted a GBA SP back then but never really saved the money and getting denied by my parents lol, maybe the RG35XXSP or 34XXSP. The former would be more versatile playing ps1 and others, but the latter would be perfect fit for GBA. Decisions, decisions. And yes, I saw that anbernic also has announced the RG SP, which is a stickless 34XXSP.
and then there's the magicx one35 my nerdery senses is killing me