A few reasons:
- The evidence so far seems to be that AI capability scales with compute, and compute continues to advance. Including with crazy AI hardware like Cerebras and Taalas. There are still a ton of AI hardware startups that have yet to tape out.
- There's an enormous amount of research into algorithmic and infrastructure improvements, which also show no signs of slowing down.
- Empirically, it has been getting better continuously for several years now. There appears to be no reason to suspect any blockers.
In other words: why not?
To steel man, the main reasons I would counter with are:
- Moore's law is pretty much dead... Except that we still have a ton of headroom with custom AI-focused chips. E.g. I know of at least two startups working on DRAM stacked on the compute chip instead of next to it, which gives dramatically higher memory bandwidth.
- There are still a couple of architectural issues, e.g. context length is limited, AI can't really remember/learn things on-line, hallucination is not solved. But... in practice those problems seem to not matter too much and also hallucinations seem less common the more capable the model is, so maybe it doesn't need an architectural solution at all.