There are conditions where sex is painful that exist for both genders. Vaginismus is maybe 6% for women and phimosis is maybe 3% for men. Numbers on these things are tough to get as many people suffer them quietly thinking they are normal.
If you or your partner experience issues it is important to know you are not alone and you have choices. That said some folks also just do not enjoy sex. Many certainly do, regardless of gender or physicallity.
If you are having issues and you don't want to, I would advise talking with someone you trust and also seeing a doctor. It is good to get a few viewpoints on issues like this as sometimes they can be dismissed.