If the populace has a bigoted plurality, then they get to declare what is officially hateful. So yes, you're right.
I put the onus on the collective citizenry, but there is no perfect solution in reality. There is a role for the state to play in protecting people, I just don't think they should dip much into what speech is or isn't allowed. The majority should rule in my opinion, but we have the job of maintaining a majority that isn't regressive bigoted shitheads. It's an eternal struggle, unfortunately.