Those papers are based on dense model architecture. The MoE architecture that I mentioned a few comments ago, does not follow the same laws. Architectural changes could push us past the wall.

Not to mention if we reach 90% accuracy (which was defined in those papers to be AGI level), there's no reason we will need to keep making new models and training them. AGI is good enough. After that we improve inference performance and bring inference cost down.