Signal doesn't use SMS anymore, and all messages are sent over encrypted Internet protocol. Any servers in between won't see the phone number, it's not needed to deliver the message, it's using an IP address at that point and the entire message metadata is encrypted. Signal is the only one that can see the phone numbers, which they use to identify multiple clients as a single user and route messages accordingly.