He's not at all invulnerable, in the book they were about to throw down and the Witch King was pretty confident about his chances. Also Gandalf literally gets killed by the Balrog. And beaten and imprisoned by Saruman.