I agree that they pose serious immediate issues regardless of thinking or not

I understand fairly well how LLMs work, but I don't think we understand the conditions for the emergence of intelligence or thought enough to conclude that "learning to predict what comes next" is not sufficient to create some form of intelligence, that's all. [Edit: if we determine that that turns out to be sufficient I will be surprised] That uncertainty is sufficient to make me agnostic on the subject and that stance does have an effect on my actions, for example I'm never mean to an LLM.

I claim ignorance as to whether they have or not, but I am fully on board with intelligence as an emergent property and believe that there is no reason a machine can't think, yes.

The American competitiveness angle is definitely.. well an interesting choice; I also was taken aback by that. Ignoring some of the posturing along those lines I find the post itself interesting.

If you think the Chinese room idea is valid we probably just fundamentally disagree and that's OK.

Odd example but yes it gets uncertain very quickly. If we discover the mechanism of our own thought it'd be easy enough to apply those rules to the new subject and see if they match or at least look similar. A negative wouldn't be conclusive, but it'd at least give us something to work with. As it stands right now we just guess, and that guess is essentially guided by bias or at best by the fact that the thing we're looking at doesn't appear even remotely similar to anything we'd call "thinking". This is simply not the case with LLMs (they do a very good job of displaying something that looks like thinking) so our guess is imo even more unsound.

I have no idea. We are unable to prove that other humans are thinking and not just "mimicking intelligence" let alone something we've created. People approaching this subject with such certainty are imo being overly confident or outright dishonest.

With respect to what I believe, since you asked, I think there is essentially no reason to distinguish "mimicking intelligence" with high fidelity from "intelligence". I don't feel comfortable talking to LLMs as if they were merely machines, but I don't approach them as if they were human either; they're in a very weird spot to me. I understand the simplicity of the math and programming behind them probably better than average, but that doesn't help me come to a conclusion on "thinking"

these models are only mimicking intelligence

I always find this to be such an interesting turn of phrase: "mimicking intelligence". If it is possible for these systems to cross the line into "thought" I think we'll miss it entirely because of our biases that the type of "thought" we have is somehow special or mystical even.

They're likely referring to https://www.anthropic.com/research/agentic-misalignment

https://www.anthropic.com/research/agentic-misalignment I believe the poster is referring to this. I wouldn't call it sensationalized

But a superintelligence quickly runs into the limits of physics and cooling becomes the bottleneck.

Can you expand on this?

Right, and regardless of sci-fi future potentials, there is a current reality and we're in it and only have so much time to affect it's evolution. All of these likely intelligent people completely ignoring reality isn't helping define and construct the world we want tomorrow.

LLMs entering the world have revealed two very different types of crazies and I'm starting to think they're everywhere. It's quite tiring to have to constantly say "no they're not useless" and "no they're not super god" instead of engage with an actually interesting discussion.