Saying eye doctor in a conversation feels normal. Saying tooth doctor feels like listening to someone who had a bad disco elysium roll and they're punching themselves in the face to get the words out.
I think the 'easier to say' part is the key thing here. "Pee-dee-atrition" is a lot easier for most English speakers that "op-tha-mologist". I think the "th" throws people off enough to just say "eye doc"