The user does decide, XDG user directories are optional and configurable. Since they are already established, user-friendly distros / desktop environments already pre-install them.
And what speaks against just using a new directory within your home directory as your "specific place that is user owned that isn't filed with cruft and configuration files"?