Well, slowness doesn't have to be severe enough to be considered intellectual disability, but the R word is still a slur. And tan skin doesn't have to be dark enough to cause racial prejudice, but the N word is still a slur. It seems that from our pre-existing examples, the answer is that if people are going to use "narcissist" as a pejorative it's a slur