I don't think people care in friendly matches, but people forget how many young girls' lives are impacted through scholarships or grants given through high school sports. When these competitions can change the course of your life, perceived fairness becomes much more important, not just for the athletes but their families as well.
My experience has been that even suggesting that someone might not be a hateful monster for questioning trans involvement in competitive sports, gets you banned from communities, and labeled a transphobe on Lemmy.