how can the Fediverse grow without loosing its soul in the process?

It can't? Rather obvious that the "soul" of a community is defined by it's members and the bigger the community the more mainstream this soul will become. Maybe the federation mechanic offers some solutions here, but that remains to be seen.