Mine has a similar problem, it would turn on under AC power but not battery. Maybe capacitors? I'm not sure I still have it anymore.

The joystick was terrible I agree. I loved the mods people did adding the Neo Geo Pocket joystick or a d-pad but they looked too difficult for me.