Because non-obvious backdoors can be added to the client that break or circumvent encryption (looking at you, xz), stealing all of your passwords, and no one will be able to raise the alarm just by looking at the server code.

Open-source backend allows to generally avoid this situation, while also potentially rendering you able to self-host if you're paranoid.