I'm not an expert but I've got a friend of a friend who works in digital security (to the point where he applied to CSIS and boy is that an interesting different story) and I was asking him about Signal.
He seemed to be very confident that Signal's code is completely public, that if it had secret backdoors or whatever people would know.
He also confirmed that while Signal may be secure (not a statement he made definitively) your phone very likely isn't, so it's a moot point.