Exactly, it opens you up to slander. They could be saying all sorts of nasty things about you to other people and you can't even defend yourself from the allegations, and no one reading it will know you had them blocked.