I think it's an instance of self-deprecating humor. Misogyny does affect queer men as well.
But there's also people who are just gross and take it to heart (to the point of sexual harassment). It doesn't really matter if you're the titled "gay friend" as long as the friend group is for queer liberation (no TERFs) and it's not some weird chaser mentality where the intrigue is objectifying queer men's bodies and sexuality.