It's easy for you and me. But imagine you're Joe Everyman. First off, you need to know about F-Droid and where to download it. Then confirm that the browser can install software. And this is where I would imagine many users second-guessing if everything's legit. And after it's installed, you've got two separate app-stores to deal with. You need to know what you can install where.
If I extrapolate from my mother-in-law, who still can't wrap her head around the concept of an app-store, let alone alternative browsers, that's just too much hassle for most people.