Not standing up for things you claim to believe in or care about makes you a hypocrite and/or a coward.
If your fear of group ostracization or abandonment outwreighs your other sense of values... you are either a coward, or, you don't actually have the values you seem to think you do.
You said it yourself, in your hypothetical, that they want to leave, which implies they comprehend that being there is bad... but they're afraid, so they don't.
How is that not cowardice?
It takes bravery to stand against a group, to critique them, and still be involved with them.
Either they don't really want to, thus they're hypocrites, or they do but they're afraid, thus they are cowards.
Perhaps a mix of both, but it has to be at least one.