Cost doesn't come down because a product gets better. Cost comes down over time as you simplify a process or as you streamline and edit it to be faster and cheaper. When it's faster/cheaper/simpler to make a product you don't spend as much on production and therefore can sell it more cheaply.

The other alternative is to make it cheaper by subsidizing the cost with something else like data collection or ads (this is what happened with televisions). This sort of loss leadership is already what's happening with Generative AI. They started off offering it free to get people in the door, offered more features for a paid plan to hook companies, and now the costs keep going up and they want to add in ads on top of the data collection they were already doing.

Business idiots will assume that once their workforce is largely replaced by AI and their company depends on it, AI vendors will make their services cheaper. When have we ever seen SaaS get cheaper over time? Why wouldn’t Anthropic or OpenAI squeeze anyone dumb enough to bet their company on it for everything they’ve got? Cost may drastically drop, although I doubt it, but those margins are for Dario and Sam and the shareholders, not the suckers.

There's good chance that the cost won't come down without new hardware. Which those companies will need to buy from scratch, trashing everything they already purchased.

or they know its slop, and they hoping to peddle enough new AI slop to keep the funding going a while longer.

The costs might come down as hardware production chains are retooled and specialized, the hardware designs become more efficient, and economies of scale catch up.

The quality won't drastically improve without some major paradigm shift in model training. All of the reasonably useful and accessible training data has already been collected and processed. There are probably still refinements to be made that will improve output quality incrementally, but after ingesting the entire Internet there's nothing left to train on. They were already starting to feed LLMs their own output as training input two years ago, which just amplifies the instabilities.

The article sucks. That doesn't change that most companies are still not even remotely cost controlling LLM usage like pretty much anyting else, nor that we even know what real costs, without billions in venture capital subsidies, would look like, other than substantially higher than now.

It’s such a time suck. It is close, so you keep trying.

It would have been faster to do it yourself but instead, you burned half a day on slop.

What did you spend 3 hours trying to fix?