Parents can be really shit to their own children! I wouldn't have believe what I realized in recent months, learning what kind of disturbed people have kids. Destructive relationships like this can even affect grand children. Probably there have been signs you didn't know how to deal with in the past. Try and find out how to leave him behind for good. It's unrealistic he will change. But I'm not a psychologist, so do some research and maybe seek help.