▲ 724 ▼ ‘The cost of compute is far beyond the costs of the employees’: Nvidia exec says right now AI is more expensive than paying human workers (fortune.com) submitted 4 months ago by Gsus4@mander.xyz to c/world@lemmy.world 106 comments fedilink hide all child comments
[–] Rooster326@programming.dev 27 points 4 months ago* (last edited 4 months ago) (3 children) Just want to point out the "more layoffs". We are still in the Uber-subsidized part of the relationship. This is the honeymoon and it ain't off to a great start permalink fedilink source parent hideshow 6 child comments replies: [–] whoisearth@lemmy.ca 12 points 4 months ago It's absolutely baffling how the gig-economy had been normalized. I am a news junkie. Just this week they were talking to a business expert about youth unemployment (note it's bad everywhere but it's starting to get bad in 1st world now so we are caring finally) and his response was yeah it's bad but basically we have to get used to a future of hustling doing multiple gigs with no health benefits because that's where things are going. Fucking government is bent over the barrel letting industry define the workers terms of engagement it's disgusting. permalink fedilink source parent [–] porous_grey_matter@lemmy.ml 3 points 4 months ago (3 children) We are still in the Uber-subsidized part of the relationship. It's not comparable, the costs for Uber are fixed development costs which decrease per customer with more customers. AI is strictly more expensive the more users there are since the cost is per use. The financial outlook for the industry is very bad, it will probably never be profitable except maybe in some extremely niche situations. permalink fedilink source parent hideshow 6 child comments replies: [–] Glemek@lemmy.world 5 points 4 months ago (1 child) I don't think they mean uber the company, I think they meant uber as in over or very. permalink fedilink source parent hideshow 2 child comments replies: [–] porous_grey_matter@lemmy.ml 3 points 4 months ago I think they mean the company since they're the most famous example of subsidising the product to gain market share and their name is invoked all the time in discussions of AI economics permalink fedilink source parent [–] Knock_Knock_Lemmy_In@lemmy.world 2 points 4 months ago (1 child) AI is strictly more expensive the more users there are since the cost is per use. Isn't most of the cost in the model training? permalink fedilink source parent hideshow 2 child comments replies: [–] porous_grey_matter@lemmy.ml 3 points 4 months ago (1 child) The cost of inference has passed the cost of training quite a while ago and it's by far the majority of expenses permalink fedilink source parent hideshow 2 child comments replies: [–] Knock_Knock_Lemmy_In@lemmy.world 1 point 4 months ago* (1 child) Thanks for the link, but disagree with your summary. The Spend on inference has passed the Spend on training. Training is the monopoly product and even the cheapest known model (deepseek) cost 10s of millions to train. permalink fedilink source parent hideshow 2 child comments replies: [–] porous_grey_matter@lemmy.ml 1 point 4 months ago (1 child) That's got nothing to do with how expensive it is to serve models to customers which is what was being discussed. permalink fedilink source parent hideshow 2 child comments replies: [–] Knock_Knock_Lemmy_In@lemmy.world 1 point 4 months ago We are discussing cost. There is fixed cost (training) and variable cost (inference). Both components being discussed. permalink fedilink source parent [–] Miaou@jlai.lu 1 point 4 months ago (1 child) Uber fares are also per use..? permalink fedilink source parent hideshow 2 child comments replies: [–] porous_grey_matter@lemmy.ml 1 point 4 months ago* Yeah, but they never really systematically subsidised fares as such, except for some one-off bonuses to attract divers and stuff like that, in the sense that they didn't systematically pay drivers more than people were paying per ride. They subsidised it in the sense that the cut they took didn't cover their development costs, servers, marketing, etc. But those costs don't increase linearly per customer and they also plateau as the software stack matures, so there's a path to just raising prices and getting to profit, and each additional customer brought them closer to profit even while the subsidies existed. OpenAI and Anthropic are paying something like 10x the amount for inference as they get from subscriptions, let alone free usage and training costs. So each new customer is taking them further from being profitable. And if they jacked up the prices 10x, so that the basic subscriptions were a few hundred and the pro ones a few thousand a month, they would still be in the position that Uber was when they were doing the subsidies for their infrastructure. I think it's fairly obvious that they wouldn't be acquiring customers very quickly at those prices. permalink fedilink source parent [–] LittleBorat3@lemmy.world 3 points 4 months ago The companies need to at least seem to grow forever. See Facebook to meta VR stuff to AI. The business models could not grow forever that's the reason for this constant reinvention of themselves. At some point AI and building these data centers will have to lead to quantum break , wow we have AGI or to monetization. Who's going to pay the actual energy costs+ margin for all their tokens? Not me permalink fedilink source parent
[–] whoisearth@lemmy.ca 12 points 4 months ago It's absolutely baffling how the gig-economy had been normalized. I am a news junkie. Just this week they were talking to a business expert about youth unemployment (note it's bad everywhere but it's starting to get bad in 1st world now so we are caring finally) and his response was yeah it's bad but basically we have to get used to a future of hustling doing multiple gigs with no health benefits because that's where things are going. Fucking government is bent over the barrel letting industry define the workers terms of engagement it's disgusting. permalink fedilink source parent
[–] porous_grey_matter@lemmy.ml 3 points 4 months ago (3 children) We are still in the Uber-subsidized part of the relationship. It's not comparable, the costs for Uber are fixed development costs which decrease per customer with more customers. AI is strictly more expensive the more users there are since the cost is per use. The financial outlook for the industry is very bad, it will probably never be profitable except maybe in some extremely niche situations. permalink fedilink source parent hideshow 6 child comments replies: [–] Glemek@lemmy.world 5 points 4 months ago (1 child) I don't think they mean uber the company, I think they meant uber as in over or very. permalink fedilink source parent hideshow 2 child comments replies: [–] porous_grey_matter@lemmy.ml 3 points 4 months ago I think they mean the company since they're the most famous example of subsidising the product to gain market share and their name is invoked all the time in discussions of AI economics permalink fedilink source parent [–] Knock_Knock_Lemmy_In@lemmy.world 2 points 4 months ago (1 child) AI is strictly more expensive the more users there are since the cost is per use. Isn't most of the cost in the model training? permalink fedilink source parent hideshow 2 child comments replies: [–] porous_grey_matter@lemmy.ml 3 points 4 months ago (1 child) The cost of inference has passed the cost of training quite a while ago and it's by far the majority of expenses permalink fedilink source parent hideshow 2 child comments replies: [–] Knock_Knock_Lemmy_In@lemmy.world 1 point 4 months ago* (1 child) Thanks for the link, but disagree with your summary. The Spend on inference has passed the Spend on training. Training is the monopoly product and even the cheapest known model (deepseek) cost 10s of millions to train. permalink fedilink source parent hideshow 2 child comments replies: [–] porous_grey_matter@lemmy.ml 1 point 4 months ago (1 child) That's got nothing to do with how expensive it is to serve models to customers which is what was being discussed. permalink fedilink source parent hideshow 2 child comments replies: [–] Knock_Knock_Lemmy_In@lemmy.world 1 point 4 months ago We are discussing cost. There is fixed cost (training) and variable cost (inference). Both components being discussed. permalink fedilink source parent [–] Miaou@jlai.lu 1 point 4 months ago (1 child) Uber fares are also per use..? permalink fedilink source parent hideshow 2 child comments replies: [–] porous_grey_matter@lemmy.ml 1 point 4 months ago* Yeah, but they never really systematically subsidised fares as such, except for some one-off bonuses to attract divers and stuff like that, in the sense that they didn't systematically pay drivers more than people were paying per ride. They subsidised it in the sense that the cut they took didn't cover their development costs, servers, marketing, etc. But those costs don't increase linearly per customer and they also plateau as the software stack matures, so there's a path to just raising prices and getting to profit, and each additional customer brought them closer to profit even while the subsidies existed. OpenAI and Anthropic are paying something like 10x the amount for inference as they get from subscriptions, let alone free usage and training costs. So each new customer is taking them further from being profitable. And if they jacked up the prices 10x, so that the basic subscriptions were a few hundred and the pro ones a few thousand a month, they would still be in the position that Uber was when they were doing the subsidies for their infrastructure. I think it's fairly obvious that they wouldn't be acquiring customers very quickly at those prices. permalink fedilink source parent
[–] Glemek@lemmy.world 5 points 4 months ago (1 child) I don't think they mean uber the company, I think they meant uber as in over or very. permalink fedilink source parent hideshow 2 child comments replies: [–] porous_grey_matter@lemmy.ml 3 points 4 months ago I think they mean the company since they're the most famous example of subsidising the product to gain market share and their name is invoked all the time in discussions of AI economics permalink fedilink source parent
[–] porous_grey_matter@lemmy.ml 3 points 4 months ago I think they mean the company since they're the most famous example of subsidising the product to gain market share and their name is invoked all the time in discussions of AI economics permalink fedilink source parent
[–] Knock_Knock_Lemmy_In@lemmy.world 2 points 4 months ago (1 child) AI is strictly more expensive the more users there are since the cost is per use. Isn't most of the cost in the model training? permalink fedilink source parent hideshow 2 child comments replies: [–] porous_grey_matter@lemmy.ml 3 points 4 months ago (1 child) The cost of inference has passed the cost of training quite a while ago and it's by far the majority of expenses permalink fedilink source parent hideshow 2 child comments replies: [–] Knock_Knock_Lemmy_In@lemmy.world 1 point 4 months ago* (1 child) Thanks for the link, but disagree with your summary. The Spend on inference has passed the Spend on training. Training is the monopoly product and even the cheapest known model (deepseek) cost 10s of millions to train. permalink fedilink source parent hideshow 2 child comments replies: [–] porous_grey_matter@lemmy.ml 1 point 4 months ago (1 child) That's got nothing to do with how expensive it is to serve models to customers which is what was being discussed. permalink fedilink source parent hideshow 2 child comments replies: [–] Knock_Knock_Lemmy_In@lemmy.world 1 point 4 months ago We are discussing cost. There is fixed cost (training) and variable cost (inference). Both components being discussed. permalink fedilink source parent
[–] porous_grey_matter@lemmy.ml 3 points 4 months ago (1 child) The cost of inference has passed the cost of training quite a while ago and it's by far the majority of expenses permalink fedilink source parent hideshow 2 child comments replies: [–] Knock_Knock_Lemmy_In@lemmy.world 1 point 4 months ago* (1 child) Thanks for the link, but disagree with your summary. The Spend on inference has passed the Spend on training. Training is the monopoly product and even the cheapest known model (deepseek) cost 10s of millions to train. permalink fedilink source parent hideshow 2 child comments replies: [–] porous_grey_matter@lemmy.ml 1 point 4 months ago (1 child) That's got nothing to do with how expensive it is to serve models to customers which is what was being discussed. permalink fedilink source parent hideshow 2 child comments replies: [–] Knock_Knock_Lemmy_In@lemmy.world 1 point 4 months ago We are discussing cost. There is fixed cost (training) and variable cost (inference). Both components being discussed. permalink fedilink source parent
[–] Knock_Knock_Lemmy_In@lemmy.world 1 point 4 months ago* (1 child) Thanks for the link, but disagree with your summary. The Spend on inference has passed the Spend on training. Training is the monopoly product and even the cheapest known model (deepseek) cost 10s of millions to train. permalink fedilink source parent hideshow 2 child comments replies: [–] porous_grey_matter@lemmy.ml 1 point 4 months ago (1 child) That's got nothing to do with how expensive it is to serve models to customers which is what was being discussed. permalink fedilink source parent hideshow 2 child comments replies: [–] Knock_Knock_Lemmy_In@lemmy.world 1 point 4 months ago We are discussing cost. There is fixed cost (training) and variable cost (inference). Both components being discussed. permalink fedilink source parent
[–] porous_grey_matter@lemmy.ml 1 point 4 months ago (1 child) That's got nothing to do with how expensive it is to serve models to customers which is what was being discussed. permalink fedilink source parent hideshow 2 child comments replies: [–] Knock_Knock_Lemmy_In@lemmy.world 1 point 4 months ago We are discussing cost. There is fixed cost (training) and variable cost (inference). Both components being discussed. permalink fedilink source parent
[–] Knock_Knock_Lemmy_In@lemmy.world 1 point 4 months ago We are discussing cost. There is fixed cost (training) and variable cost (inference). Both components being discussed. permalink fedilink source parent
[–] Miaou@jlai.lu 1 point 4 months ago (1 child) Uber fares are also per use..? permalink fedilink source parent hideshow 2 child comments replies: [–] porous_grey_matter@lemmy.ml 1 point 4 months ago* Yeah, but they never really systematically subsidised fares as such, except for some one-off bonuses to attract divers and stuff like that, in the sense that they didn't systematically pay drivers more than people were paying per ride. They subsidised it in the sense that the cut they took didn't cover their development costs, servers, marketing, etc. But those costs don't increase linearly per customer and they also plateau as the software stack matures, so there's a path to just raising prices and getting to profit, and each additional customer brought them closer to profit even while the subsidies existed. OpenAI and Anthropic are paying something like 10x the amount for inference as they get from subscriptions, let alone free usage and training costs. So each new customer is taking them further from being profitable. And if they jacked up the prices 10x, so that the basic subscriptions were a few hundred and the pro ones a few thousand a month, they would still be in the position that Uber was when they were doing the subsidies for their infrastructure. I think it's fairly obvious that they wouldn't be acquiring customers very quickly at those prices. permalink fedilink source parent
[–] porous_grey_matter@lemmy.ml 1 point 4 months ago* Yeah, but they never really systematically subsidised fares as such, except for some one-off bonuses to attract divers and stuff like that, in the sense that they didn't systematically pay drivers more than people were paying per ride. They subsidised it in the sense that the cut they took didn't cover their development costs, servers, marketing, etc. But those costs don't increase linearly per customer and they also plateau as the software stack matures, so there's a path to just raising prices and getting to profit, and each additional customer brought them closer to profit even while the subsidies existed. OpenAI and Anthropic are paying something like 10x the amount for inference as they get from subscriptions, let alone free usage and training costs. So each new customer is taking them further from being profitable. And if they jacked up the prices 10x, so that the basic subscriptions were a few hundred and the pro ones a few thousand a month, they would still be in the position that Uber was when they were doing the subsidies for their infrastructure. I think it's fairly obvious that they wouldn't be acquiring customers very quickly at those prices. permalink fedilink source parent
[–] LittleBorat3@lemmy.world 3 points 4 months ago The companies need to at least seem to grow forever. See Facebook to meta VR stuff to AI. The business models could not grow forever that's the reason for this constant reinvention of themselves. At some point AI and building these data centers will have to lead to quantum break , wow we have AGI or to monetization. Who's going to pay the actual energy costs+ margin for all their tokens? Not me permalink fedilink source parent