973
Managers (thelemmy.club)
submitted 4 months ago* by to c/whitepeopletwitter@sh.itjust.works
180 comments fedilink hide all child comments
 
Managers
you are viewing a single comment's thread
view the rest of the comments
[–] 2 points 4 months ago (1 child)

AI companies are bankrupting themselves with training costs that they need to recoup back by selling inference.

I think they hit a wall in actual returns on performance with pretraining, years ago. Then they started scaling up on post-training/reinforcement learning to continue improvement, but that might be hitting a plateau as well. More recently it looks like they're relying more heavily on scaling up on inference, which is a significant problem for their long term business models.

If they're not able to cheaply deliver inference (and charge at a premium), how will they be able to sustain their businesses?

It seems that the most recent, largest models are using a lot more tokens to accomplish the same tasks, so even as token cost drops the actual cost of using the latest models seems to be going up with time (even as performance improves).

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 1 point 4 months ago*

    If they’re not able to cheaply deliver inference (and charge at a premium), how will they be able to sustain their businesses?

    I definitely agree that they have a big problem on their hands, and are in deep deep trouble. They are in a position where they must sell a service that is very cheap in order to pay for up front costs that were very expensive.

    This is also why the release of Deepseek was such a devastating blow to US AI companies. It proved that:

    1. they don't really have a moat that would lock users into their service, or secret special knowledge that prevents other companies from training competitive models. They're in a race to the bottom

    2. Deepseek was not only able to train a model of the same caliber, but they were able to do it at a tiny fraction of the cost that US AI companies spent on training US models. Because they spent so much less on training, it means that Deepseek is able to undercut the US companies and offer inference at a much lower price

  • permalink
  • fedilink
  • source
  • parent