There's a few tickets floating around regarding the user flow for how blocking users actually works. This is probably one of those untested cases where you've blocked someone but you can see see them when they do X.

The blocking workflow is definitely something that needs some improvements, but I think right now everyone is focused on stability (especially with the influx of new people everyone's expecting)