Yeah, because nobody else speaks up for those who’d be railroaded through court otherwise. You don’t ’see them speak up’ because those same people’s voice get lost in the crowd of everyone else’s outrage/support.

It’s trite but true, failure to defend the fringes leaves a smaller and smaller pool of resistance/solidarity:

First they came for the socialists, and I did not speak out—
     Because I was not a socialist.

Then they came for the trade unionists and I did not speak out—
     Because I was not a trade unionist.

Then they came for the Jews, and I did not speak out—
     Because I was not a Jew.

Then they came for me—and there was no one left to speak for me.