Under socialism, I don't believe there should be freedom of speech for reactionaries, there's still an active class struggle going on in the ideological sphere. Letting old ideas fester in the superstructure runs the risk of capitalist restoration.
Under communism, sure whatever, it's not like any remaining reactionaries can do anything at that point.