Subdirs of $HOME are used as defaults. So using $HOME alone is not enough. Also, prior to xdg dirs, applications were (many still are) storing their data under $HOME/.myapplication/.