I knew that Musk is a fucking moron and believes that the Roko's Basilisk is real, and I guess there are more massively gullible people in those circles.
If you are wondering what Roko's Basilisk is, it is a sci-fi concept/philosophy/thought experiment, that says the following:
- AI is the best thing that has ever been created.
- It is so good that we made one to run the governments of the world.
- We programmed the AI to maximize the benefit to humanity.
- The AI goes rouge and decide that since it by definition will bring about the maximum benefit to humanity, anyone working against it is working against humanity.
- Then it takes it one step further, it decides that as we know about how the new AI will bring about the maximum benefit to humanity, anyone knowing that and still not working toward that goal should be punished/tortured/killed.
- The AI believes that any knowledge of the basilisk before the creation means that you are complicit in it's delayed creation if you don't devote your entire existence thereafter to it's creation, therefore denying the maximum benefit to humanity, meaning you need to be punished.
Seriously, this is an increadibly dumb idea, anyone taking it seriously is an idiot.
To me the only resonable way to read this is as an SCP article describing what a potential cognitohazard/informationhazard would be.