It's FOSS and a pornographic game, but I don't think that it really fits OP's criteria:

I’ve played a few Renpy graphic novels, but all are flat mostly lifeless grinders. Probably everything I have ever played is on par with the crap I talked about with school friends in 3rd to 5th grade school boy lunch cringe talk. I like novel complexity. I do not find conservative convention or stereotypes amusing, and despise typical Western misogynistic roles and expectations.

Degrees of Lewdity has the player playing a kid being sexually-abused, and has one interacting with other kids who are sexually-abused. The game permits changing genders of characters, so I guess technically one could have male characters instead of female characters in that role, but I think that it'd fail the misogyny criteria. Humiliation and degradation are kind of major elements of the game.

Free Cities is another successful erotic FOSS game with a fair bit of writing, albeit not really primarily plot-oriented, but again, really falls over on the misogyny criteria; the game is about sexual enslavement in a dystopia.

Honestly, I'd have a hard time thinking of much non-FOSS material, much less FOSS, that qualifies.

For any material, FOSS or no, a lot of pornographic content is what I'd call not all that sophisticated. Someone's aroused, they aren't necessarily going for a lot of complexity. So a lot of erotic video games aren't going to have terribly sophisticated plots to begin with.

For FOSS in particular, I think that there are very few plot-oriented games, adult or no. The problem is that most FOSS developers want to be able to enjoy the games they write themselves, and...it's hard to do that with plot. FOSS does well with things like roguelikes, where there's a procedurally-generated aspect to the game, rather than plot-oriented stuff.

And on top of that, I can think of very few games that have "novel complexity". Video games in general don't really have "noveI complexity". I guess text-heavy stuff like visual novels or interactive fiction come the closest.

The only explicitly-FOSS adult visual novel that I can think of off the top of my head is Katawa Shoujo, and I don't know if it'd meet the author's criteria. That's not to say that there aren't other adult VNs out there under open-source licenses, but I think that most are simply released freely; I don't see much discussion of license, for whatever reason, in the VN world.

Kinda similar for interactive fiction. There's plenty of freely-available interactive fiction out there, including some with erotic content. But most of it isn't open-source.

SugarCube

a popular language for authoring Twine games

can be decompiled with tweego, so one can generally access source code, but that's not the same thing as being FOSS. I haven't seen much by way of explicitly-FOSS Twine games (Degrees of Lewdity and Free Cities being exceptions, erotic Twine FOSS games).

I think that maybe with heavily plot-based stuff, developers don't feel the point of collaboration, which is kind of a major reason for making something FOSS. Most novels aren't written by a team.

If OP's willing to simply permit for non-open-source, freely-available content, that would help free things up considerably.

EDIT: Oh, Fort of Chains, intended to be

I'm not sure it succeeds

an open-source take on No Haven. That's Twine, and open-source. I don't think that it'd meet a "novel length complexity" constraint either.

At some point Corruption of Champions was open-sourced, though IIRC that's authored in some old version of Actionscript, and I don't know how easy building and running it in 2025 is going to be, and while that's got writing, I think that "novel complexity" is also going to be out of reach there.