It's a rare example of English being simpler than other languages, so I'm curious if it's hard for a new speaker to keep the nouns straight without the extra clues.
For the most part I don't think about it at all. I guess you only consider things when they cause extra effort, in this case it mostly doesn't so it's very unconscious. That said, I generally use the few gendered ones I know (I listed in another comment) because it is the way my native language works.
By the way, from grammar perspective English is a very simple language. Compared to similar languages (French, Italian etc.), for example, verbs are much simpler too.
The harder part of English I think has to do with pronounce.