▲ 891 ▼ Nvidia falls 14% in premarket trading as China's DeepSeek triggers global tech sell-off (www.cnbc.com) submitted 2 years ago by schizoidman@lemm.ee to c/technology@lemmy.world 329 comments fedilink hide all child comments cross-posted from: https://lemm.ee/post/53805638
[+] theunknownmuncher@lemmy.world 15 points 2 years ago (1 child) [deleted] permalink fedilink source parent hideshow 2 child comments replies: [–] jlh@lemmy.jlh.name 9 points 2 years ago (1 child) Ah, fair. I guess it makes sense that Wall Street is questioning the need for these expensive blackwell gpus when the hopper gpus are already so good? permalink fedilink source parent hideshow 2 child comments replies: [–] legion02@lemmy.world 7 points 2 years ago (1 child) It's more that the newer models are going to need less compute to train and run them. permalink fedilink source parent hideshow 2 child comments replies: [–] frezik@midwest.social 10 points 2 years ago (1 child) Right. There's indications of 10x to 100x less compute power needed to train the models to an equivalent level. Not a small thing at all. permalink fedilink source parent hideshow 2 child comments replies: [–] NuXCOM_90Percent@lemmy.zip 5 points 2 years ago* Not small but... smaller than you would expect. Most companies aren't, and shouldn't be, training their own models. Especially with stuff like RAG where you can use the highly trained model with your proprietary offline data with only a minimal performance hit. What matters is inference and accuracy/validity. Inference being ridiculously cheap (the reason why AI/ML got so popular) and the latter being a whole different can of worms that industry and researchers don't want you to think about (in part because "correct" might still be blatant lies because it is based on human data which is often blatant lies but...). And for the companies that ARE going to train their own models? They make enough bank that ordering the latest Box from Jensen is a drop in the bucket. That said, this DOES open the door back up for tiered training and the like where someone might use a cheaper commodity GPU to enhance an off the shelf model with local data or preferences. But it is unclear how much industry cares about that. permalink fedilink source parent
[–] jlh@lemmy.jlh.name 9 points 2 years ago (1 child) Ah, fair. I guess it makes sense that Wall Street is questioning the need for these expensive blackwell gpus when the hopper gpus are already so good? permalink fedilink source parent hideshow 2 child comments replies: [–] legion02@lemmy.world 7 points 2 years ago (1 child) It's more that the newer models are going to need less compute to train and run them. permalink fedilink source parent hideshow 2 child comments replies: [–] frezik@midwest.social 10 points 2 years ago (1 child) Right. There's indications of 10x to 100x less compute power needed to train the models to an equivalent level. Not a small thing at all. permalink fedilink source parent hideshow 2 child comments replies: [–] NuXCOM_90Percent@lemmy.zip 5 points 2 years ago* Not small but... smaller than you would expect. Most companies aren't, and shouldn't be, training their own models. Especially with stuff like RAG where you can use the highly trained model with your proprietary offline data with only a minimal performance hit. What matters is inference and accuracy/validity. Inference being ridiculously cheap (the reason why AI/ML got so popular) and the latter being a whole different can of worms that industry and researchers don't want you to think about (in part because "correct" might still be blatant lies because it is based on human data which is often blatant lies but...). And for the companies that ARE going to train their own models? They make enough bank that ordering the latest Box from Jensen is a drop in the bucket. That said, this DOES open the door back up for tiered training and the like where someone might use a cheaper commodity GPU to enhance an off the shelf model with local data or preferences. But it is unclear how much industry cares about that. permalink fedilink source parent
[–] legion02@lemmy.world 7 points 2 years ago (1 child) It's more that the newer models are going to need less compute to train and run them. permalink fedilink source parent hideshow 2 child comments replies: [–] frezik@midwest.social 10 points 2 years ago (1 child) Right. There's indications of 10x to 100x less compute power needed to train the models to an equivalent level. Not a small thing at all. permalink fedilink source parent hideshow 2 child comments replies: [–] NuXCOM_90Percent@lemmy.zip 5 points 2 years ago* Not small but... smaller than you would expect. Most companies aren't, and shouldn't be, training their own models. Especially with stuff like RAG where you can use the highly trained model with your proprietary offline data with only a minimal performance hit. What matters is inference and accuracy/validity. Inference being ridiculously cheap (the reason why AI/ML got so popular) and the latter being a whole different can of worms that industry and researchers don't want you to think about (in part because "correct" might still be blatant lies because it is based on human data which is often blatant lies but...). And for the companies that ARE going to train their own models? They make enough bank that ordering the latest Box from Jensen is a drop in the bucket. That said, this DOES open the door back up for tiered training and the like where someone might use a cheaper commodity GPU to enhance an off the shelf model with local data or preferences. But it is unclear how much industry cares about that. permalink fedilink source parent
[–] frezik@midwest.social 10 points 2 years ago (1 child) Right. There's indications of 10x to 100x less compute power needed to train the models to an equivalent level. Not a small thing at all. permalink fedilink source parent hideshow 2 child comments replies: [–] NuXCOM_90Percent@lemmy.zip 5 points 2 years ago* Not small but... smaller than you would expect. Most companies aren't, and shouldn't be, training their own models. Especially with stuff like RAG where you can use the highly trained model with your proprietary offline data with only a minimal performance hit. What matters is inference and accuracy/validity. Inference being ridiculously cheap (the reason why AI/ML got so popular) and the latter being a whole different can of worms that industry and researchers don't want you to think about (in part because "correct" might still be blatant lies because it is based on human data which is often blatant lies but...). And for the companies that ARE going to train their own models? They make enough bank that ordering the latest Box from Jensen is a drop in the bucket. That said, this DOES open the door back up for tiered training and the like where someone might use a cheaper commodity GPU to enhance an off the shelf model with local data or preferences. But it is unclear how much industry cares about that. permalink fedilink source parent
[–] NuXCOM_90Percent@lemmy.zip 5 points 2 years ago* Not small but... smaller than you would expect. Most companies aren't, and shouldn't be, training their own models. Especially with stuff like RAG where you can use the highly trained model with your proprietary offline data with only a minimal performance hit. What matters is inference and accuracy/validity. Inference being ridiculously cheap (the reason why AI/ML got so popular) and the latter being a whole different can of worms that industry and researchers don't want you to think about (in part because "correct" might still be blatant lies because it is based on human data which is often blatant lies but...). And for the companies that ARE going to train their own models? They make enough bank that ordering the latest Box from Jensen is a drop in the bucket. That said, this DOES open the door back up for tiered training and the like where someone might use a cheaper commodity GPU to enhance an off the shelf model with local data or preferences. But it is unclear how much industry cares about that. permalink fedilink source parent