"Low" in this clip is describing social classes and "company" here just means the people someone associates with. He is worried that "polite society" will shun his family because of what he did and they will be forced to associate with people who would be considered lower class, like criminals, to survive.