It's worth noting that mutants are often the ones making distinctions, even the X-Men..

If you're born with powers but don't carry the x-gene they'll be the first ones to tell you you can't be in their club, even if you used to be in it when they thought you carried it (see Wanda and Pietro Maximoff or Franklin Richards for notorious examples).