This has been my experience, too. They judge me for it but they don't realize I'm awake for the same amount of time they are, just not at the same time.

Frankly, I don't miss the human interaction that I'm not getting by staying up so late.