No, apps (at least on Android) require a specific permission to read your phone number. The phone capabilities of Signal go through their own VoIP service, not your phone.