I mean the meme caption doesn't say "a street full of grown men AND a bunch of women who aren't looking"

I think you can point out that one gender typically enjoys something without automatically implying another gender doesn't. Or at least we should be able to