Take Signal to Matrix for example. They use different encryption protocols, which means a message sent from one end has to be decrypted, and then re-encrypted with the protocol of the recipient before they can actually receive it.

So basically, your encryption is not very e2e anymore, and the fact that someone can set this up, effectively giving encryption keys to a third party without their contacts being able to do anything about it is pretty fucked.

Oh, and different TOS between different services also come into play.

So if you do this, at least tell your contacts about it, so they can make an informed decision about whether or not that's okay for them.