the_dunk_tank

I haven't seen anyone here (or basically anyone at all, for that matter) suggest that there's literally no way to create mentality like ours other than being exactly like us. The argument is just that LLMs are not even on the right track to do something like that. The technology is impressive in a lot of ways, but it is in no way comparable to even a rudimentary mind in the sense that people have minds, and there's no amount of tweaking or refining the basic approach that's going to move it in that direction. "Genuine" (in the sense of human-like) AI made from non-human stuff is certainly possible in principle, but LLMs are not even on that trajectory.

Even setting that aside, I think framing this as an I/O problem elides some really tricky and deep conceptual content, and suggests some fundamental misunderstanding about how complex this problem is. What on Earth does "the output of human thought" mean in this sense? Clearly you don't really mean human thought, because you obviously think whatever "output" you're looking for can be instantiated in non-human systems. It must mean human-like thought, but human-like in what sense? Which features are important to preserve, and which are incidental or parochial to the way humans do human-like thought? How you answer that question greatly influences how you evaluate putative cases of "genuine" AI, and it's possible to build in a great deal of hidden bias if we don't think carefully and deliberately about this. From what I've seen, virtually none of the AI hypers are thinking carefully or deliberately about this.