There is actually a hallucinogenic plant that looks kind of similar to mint, but I think they're referring to the fact that mint chokes other plants out and just sticks around and keeps coming back.

They spread and are really really hard to fully kill

Source: I have no idea why my mint is still alive. It's waterlogged for half the year and neglected the other half

Exactly: neighbors can stay mad. Mint is cooler than neighbors arguably (& chemically).

Not to mention that mosquitos hate mint.