AI companies are bankrupting themselves with training costs that they need to recoup back by selling inference.

I think they hit a wall in actual returns on performance with pretraining, years ago. Then they started scaling up on post-training/reinforcement learning to continue improvement, but that might be hitting a plateau as well. More recently it looks like they're relying more heavily on scaling up on inference, which is a significant problem for their long term business models.

If they're not able to cheaply deliver inference (and charge at a premium), how will they be able to sustain their businesses?

It seems that the most recent, largest models are using a lot more tokens to accomplish the same tasks, so even as token cost drops the actual cost of using the latest models seems to be going up with time (even as performance improves).

Inference is cheap and efficient.

Tell that to all the Github users that are screaming about the new token based billing. In reality inference on these massive models with big context windows is expensive, but was subsidized so hard, that nobody has an accurate feeling for the cost.