Funnily enough, the labor to create something is a large basis of Proudhon's critique in "What is Property?" You sure as hell didn't spend labor to create the land that house was built on. No one did. Who gave you control over it, and how was it their right?