that defeats the purpose of a package manager

I suspect you're confusing the package manager with the supply chain. This should always be separate, lest we get some systemd-like spongiform blob with no ability to swap out and improve parts without discarding the whole.

Unless you're Lennart, it's a bad idea.