I think this is kind of a thing for a lot of stereoscopic 3d technology.

I could play the virtual boy, 3ds, watch 3d movies, etc. for hours without issue, and other people can't take it at all.

I don't know what factors play into that, maybe it's genetic, maybe there's some kind of skill/technique/habits about how you focus your eyes, or how often you blink, maybe it's just luck of the draw that your pupil distance is just right or wrong. Maybe it comes down to something ridiculous like how many hours you spent trying to make sense of Magic Eye books when you were a kid.