Debian would not create and maintain a "core debian" variant just to be installed then receive the extra packages
Debian server minimal, is kind of a "core Debian". There are netinst versions that can be even smaller. The Debian base image for Docker is even smaller than all that.
There is also an Ubuntu minimal install that you could call "core Ubuntu".
But more importantly, and I can't stress this enough: YOU CAN SWITCH DISTROS WITHOUT REINSTALLING. Might need to do some cleanup afterwards, but it's perfectly doable, more so between Debian based ones.