Epistatus: I skimmed the first 20 pages of the paper. I have not watched the WoW documentary.
Yeah, the paper doesn’t seem to do a good job at justifying introducing system 3. They should have at least looked at why you couldn’t model this as something like:
- Person encounters a situation requiring thought
- System 1 triggers System 2
- System 2 offloads cognition to some black box
- The result is absorbed and successfully addresses the situation
- A- System 1 is rewarded and begins adding use of the black box to its repertoire, skipping System 2
or
B- System 2 absorbs the result and begins trusting the black box more, perhaps deferring to its result before any internal cognition
Both of these things I pulled outta my ass can explain deskilling, and a situation like 5a) might explain capitulating to AI under pressure. So I’m not feeling like System 3 is necessary here.
That all being said, it’s not like the paper gets to decide how the rest of the world chooses to use “cognitive surrender”.