Of course closed source software can be secure, but given that you can't verify that it actually is, it's better to treat it as if it's not. And I'm not saying that WhatsApp isn't encrypted, but we can't know if this also applies properly to metadata, or which encryption algorithm is actually used, if the keys are really only accessible to end users (i.e. if it's really E2EE), etc.

It's not unreasonable to think that Meta (or any other Big Tech company) makes their products just secure enough to not allow just about anyone with the right abilities to exploit weaknesses (which would obviously hurt their business), while still retaining the ability for themselves to have access to at least some of the data.

Also, how is it a good idea to give a signal my contacts if it's not okay to give meta my contacts?

Also, if you don't accept this argument, then how do you know there's not a component in your phone that listens to everything you say that's above system level?...

You seem to be treating privacy like an all-or-nothing situation. There's different levels to privacy, with different realistic possibilities and threat models. To give you the short answers to both questions: It's not a good idea to give Signal access to your contacts, and I indeed don't know if there's a component in my phone spying on everything. But there is nuance to both.

First off, Signal's Contact Dicovery Service is open source. Of course there's still a level of trust required regarding Signal's operational integrity, as we can't know if this is really what runs on their servers, but at least the Signal Foundation doesn't have a history of countless known privacy violations like Meta does, which means it all comes down to how much you trust either of those organisations. So even though I wouldn't recommend granting Signal access to your contacts and personally prefer messengers without such a feature (e.g. SimpleX), I also don't yet insist on recommending against it.

As for the hardware: Assuming that there is indeed some component spying on me, why would this be a justification to become careless about privacy on the software level? And the real difference between the arguments is the availability of alternatives. While there are quite a few open-source messenger alternatives, the same can't be said about hardware. And it's unlikely for there to be anything on the hardware level that could be spying without anyone noticing, at least if we assume that this hardware would send this data somewhere, which one would surely notice when analysing network traffic or by checking for emitted radio frequencies if the manufacturer had the genius idea of using mesh networks instead.
However, certain components could certainly be used as a backdoor for state actors if they gain physical access to a device, so I'm still advocating for open hardware and firmware.