Private boycotts do not have material consequences. Big loud public ones sometimes do.
In other words private boycotts DO have material consequences, as long as they're mediated through collective action, I'd like to see how you organize these collective boycotts without the individual actions of private boycotters and the appropriate moral character that brings light to the suffering of child laborers. Is there a psychic beacon that magically convinces everyone big boycotts are worthwhile, or maybe it has to start somewhere?
This is why egoist arguments like yours are incoherent at scale, instead of merely viewing morality as a religious phenomenon, you should view it in part as a powerful signaling mechanism that can solidify an underlying collective energy, which serves as the catalyst for the material transformation you seemingly care about