In the Flashback episode it is noted their house is smaller then their previous apartment. It also shows how they went from comfortable/upscale living to cheaper conditions with each additional child, specifically stating they had less money with each birth.