I think that's a problem with jury trials in general, and even authority in general. Sometimes people who you disagree with strongly end up with the power to affect the lives of others. Jury powers, just as any others, can be abused. But they can also check abuses, like if a prosecutor is overzealous, a judge is biased, or lawmakers corrupt.

I think the best we can do is spread power out as much as possible and hope that people will ultimately prove to be good more often than bad, so that all of the maneuvering and countering people use their power for will trend more towards good. Any solution that concentrates power to keep it out if the hands of bad people has the footnote "assuming those who get that power are good and use it in good faith and don't change while they have power", which is not a safe assumption.

What happens when the person's principles include hating women

You're describing cops.

I was shot by a fascist. 11 people believed the evidence, one person was also a fascist. Did anyone need to tell the fascist about jury nullification? No. They don't care about the law and never have. That's the point. They already do it all the time. You should know that you can do it too.