Part of the problem is that from a social history standpoint, libertarianism typically has attracted people looking for an ideology to justify their selfishness.

The ideology that tends to attract people who value social organization while minimizing a forceful overarching government has been anarchism.