In capitalism as inventory goes down prices go up. As we descend upon that small town and rent or buy what's there, prices go up.We need to instead lobby for legislation that heavily taxes investment properties. That will free the hundreds of thousands of homes that're empty. Also local housing initiatives where tax dollars go to subsidize the building of more houses. Look at the G.I. Bill after WW2 for a great example of this. But instead of veterans, anyone with income 90k or below gets a check to help with their first home. Of course we'd keep the GI Bill for veterans as well. This isn't a zero sum game.

This is called gentrification.