The difficulty of getting ferrystones is by design, no different than the previous game. If you fast travel everywhere you're missing out on a lot of the events in the world. They're handy to use in certain situations, but there are other options to get around.