▲ 891 ▼ Nvidia falls 14% in premarket trading as China's DeepSeek triggers global tech sell-off (www.cnbc.com) submitted 2 years ago by schizoidman@lemm.ee to c/technology@lemmy.world 329 comments fedilink hide all child comments cross-posted from: https://lemm.ee/post/53805638
[+] ArchRecord@lemm.ee 17 points 2 years ago* (last edited 1 year ago) (4 children) [deleted] permalink fedilink source parent hideshow 8 child comments replies: [–] aesthelete@lemmy.world 21 points 2 years ago* Thank the fucking sky fairies actually, because even if AI continues to mostly suck it'd be nice if it didn't swallow up every potable lake in the process. When this shit is efficient that makes it only mildly annoying instead of a complete shitstorm of failure. permalink fedilink source parent [–] adoxographer@lemmy.world 6 points 2 years ago (3 children) While this is great, the training is where the compute is spent. The news is also about R1 being able to be trained, still on an Nvidia cluster but for 6M USD instead of 500 permalink fedilink source parent hideshow 6 child comments replies: [–] alvvayson@lemmy.dbzer0.com 5 points 2 years ago (2 children) True, but training is one-off. And as you say, a factor 100x less costs with this new model. Therefore NVidia just saw 99% of their expected future demand for AI chips evaporate Even if they are lying and used more compute, it's obvious they managed to train it without access to the large amounts of the highest end chips due to export controls. Conservatively, I think NVidia is definitely going to have to scale down by 50% and they will have to reduce prices by a lot, too, since VC and government billions will no longer be available to their customers. permalink fedilink source parent hideshow 4 child comments replies: [–] bestboyfriendintheworld@sh.itjust.works 3 points 2 years ago (1 child) True, but training is one-off. And as you say, a factor 100x less costs with this new model. Therefore NVidia just saw 99% of their expected future demand for AI chips evaporate It might also lead to 100x more power to train new models. permalink fedilink source parent hideshow 2 child comments replies: [+] ArchRecord@lemm.ee 1 point 2 years ago* (last edited 1 year ago) (1 child) [deleted] permalink fedilink source parent hideshow 2 child comments replies: [–] bestboyfriendintheworld@sh.itjust.works 2 points 2 years ago Thank you. Sounds like good news. permalink fedilink source parent [–] adoxographer@lemmy.world 3 points 2 years ago (1 child) I’m not sure. That’s a very static view of the context. While china has an AI advantage due to wider adoption, less constraints and overall bigger market, the US has higher tech, and more funds. OpenAI, Anthropic, MS and especially X will all be getting massive amounts of backing and will reverse engineer and adopt whatever advantages R1 had. Which while there are some it’s still not a full spectrum competitor. I see the is as a small correction that the big players will take advantage of to buy stock, and then pump it with state funds, furthering the gap and ignoring the Chinese advances. Regardless, Nvidia always wins. They sell the best shovels. In any scenario the world at large still doesn’t have their Nvidia cluster, think Africa, Oceania, South America, Europe, SEA who doesn’t necessarily align with Chinese interests, India. Plenty to go around. permalink fedilink source parent hideshow 2 child comments replies: [–] alvvayson@lemmy.dbzer0.com 2 points 2 years ago (1 child) Extra funds are only useful if they can provide a competitive advantage. Otherwise those investments will not have a positive ROI. The case until now was built on the premise that US tech was years ahead and that AI had a strong moat due to high computer requirements for AI. We now know that that isn't true. If high compute enables a significant improvement in AI, then that old case could become true again. But the prospects of such a reality happening and staying just got a big hit. I think we are in for a dot-com type bubble burst, but it will take a few weeks to see if that's gonna happen or not. permalink fedilink source parent hideshow 2 child comments replies: [–] adoxographer@lemmy.world 1 point 2 years ago Maybe, but there is incentive to not let that happen, and I wouldn’t be surprised if “they” have unpublished tech that will be rushed out. The ROI doesn’t matter, it wasn’t there yet it’s the potential for it. The Chinese AIs are also not there yet. The proposition is to reduce FTEs, regardless of cost, as long as cost is less. While I see OpenAi and mostly startups and VC reliant companies taking a hit, Nvidia itself as the shovel maker will remain strong. permalink fedilink source parent [–] vrighter@discuss.tchncs.de 2 points 2 years ago if, on a modern gaming pc, you can get breakneck speeds of 5 tokens per second, then actually inference is quite energy intensive too. 5 per second of anything is very slow permalink fedilink source parent [–] orange@communick.news 2 points 2 years ago (1 child) That's becoming less true. The cost of inference has been rising with bigger models, and even more so with "reasoning models". Regardless, at the scale of 100M users, big one-off costs start looking small. permalink fedilink source parent hideshow 2 child comments replies: [–] adoxographer@lemmy.world 1 point 2 years ago (1 child) But I’d imagine any Chinese operator will handle scale much better? Or? permalink fedilink source parent hideshow 2 child comments replies: [–] orange@communick.news 1 point 2 years ago Maybe? Depends on what costs dominate operations. I imagine Chinese electricity is cheap but building new data centres is likely much cheaper % wise than countries like the US. permalink fedilink source parent [–] alvvayson@lemmy.dbzer0.com 3 points 2 years ago (1 child) Wth?! Like seriously. I assume they are running the smallest version of the model? Still, very impressive. permalink fedilink source parent hideshow 2 child comments replies: [–] vrighter@discuss.tchncs.de 3 points 2 years ago i can also run it on my old pentium from 3 decades ago. I'd have to swap 4MiB of weights in and out constantly, it will be very very slow, but it will work. permalink fedilink source parent [–] GenosseFlosse@feddit.org 0 points 2 years ago (1 child) Sure you can run it on low end hardware, but how does the performance (response time for a given prompt) compare to the other models, either local or as a service? permalink fedilink source parent hideshow 2 child comments replies: [+] ArchRecord@lemm.ee 1 point 2 years ago* (last edited 1 year ago) [deleted] permalink fedilink source parent
[–] aesthelete@lemmy.world 21 points 2 years ago* Thank the fucking sky fairies actually, because even if AI continues to mostly suck it'd be nice if it didn't swallow up every potable lake in the process. When this shit is efficient that makes it only mildly annoying instead of a complete shitstorm of failure. permalink fedilink source parent
[–] adoxographer@lemmy.world 6 points 2 years ago (3 children) While this is great, the training is where the compute is spent. The news is also about R1 being able to be trained, still on an Nvidia cluster but for 6M USD instead of 500 permalink fedilink source parent hideshow 6 child comments replies: [–] alvvayson@lemmy.dbzer0.com 5 points 2 years ago (2 children) True, but training is one-off. And as you say, a factor 100x less costs with this new model. Therefore NVidia just saw 99% of their expected future demand for AI chips evaporate Even if they are lying and used more compute, it's obvious they managed to train it without access to the large amounts of the highest end chips due to export controls. Conservatively, I think NVidia is definitely going to have to scale down by 50% and they will have to reduce prices by a lot, too, since VC and government billions will no longer be available to their customers. permalink fedilink source parent hideshow 4 child comments replies: [–] bestboyfriendintheworld@sh.itjust.works 3 points 2 years ago (1 child) True, but training is one-off. And as you say, a factor 100x less costs with this new model. Therefore NVidia just saw 99% of their expected future demand for AI chips evaporate It might also lead to 100x more power to train new models. permalink fedilink source parent hideshow 2 child comments replies: [+] ArchRecord@lemm.ee 1 point 2 years ago* (last edited 1 year ago) (1 child) [deleted] permalink fedilink source parent hideshow 2 child comments replies: [–] bestboyfriendintheworld@sh.itjust.works 2 points 2 years ago Thank you. Sounds like good news. permalink fedilink source parent [–] adoxographer@lemmy.world 3 points 2 years ago (1 child) I’m not sure. That’s a very static view of the context. While china has an AI advantage due to wider adoption, less constraints and overall bigger market, the US has higher tech, and more funds. OpenAI, Anthropic, MS and especially X will all be getting massive amounts of backing and will reverse engineer and adopt whatever advantages R1 had. Which while there are some it’s still not a full spectrum competitor. I see the is as a small correction that the big players will take advantage of to buy stock, and then pump it with state funds, furthering the gap and ignoring the Chinese advances. Regardless, Nvidia always wins. They sell the best shovels. In any scenario the world at large still doesn’t have their Nvidia cluster, think Africa, Oceania, South America, Europe, SEA who doesn’t necessarily align with Chinese interests, India. Plenty to go around. permalink fedilink source parent hideshow 2 child comments replies: [–] alvvayson@lemmy.dbzer0.com 2 points 2 years ago (1 child) Extra funds are only useful if they can provide a competitive advantage. Otherwise those investments will not have a positive ROI. The case until now was built on the premise that US tech was years ahead and that AI had a strong moat due to high computer requirements for AI. We now know that that isn't true. If high compute enables a significant improvement in AI, then that old case could become true again. But the prospects of such a reality happening and staying just got a big hit. I think we are in for a dot-com type bubble burst, but it will take a few weeks to see if that's gonna happen or not. permalink fedilink source parent hideshow 2 child comments replies: [–] adoxographer@lemmy.world 1 point 2 years ago Maybe, but there is incentive to not let that happen, and I wouldn’t be surprised if “they” have unpublished tech that will be rushed out. The ROI doesn’t matter, it wasn’t there yet it’s the potential for it. The Chinese AIs are also not there yet. The proposition is to reduce FTEs, regardless of cost, as long as cost is less. While I see OpenAi and mostly startups and VC reliant companies taking a hit, Nvidia itself as the shovel maker will remain strong. permalink fedilink source parent [–] vrighter@discuss.tchncs.de 2 points 2 years ago if, on a modern gaming pc, you can get breakneck speeds of 5 tokens per second, then actually inference is quite energy intensive too. 5 per second of anything is very slow permalink fedilink source parent [–] orange@communick.news 2 points 2 years ago (1 child) That's becoming less true. The cost of inference has been rising with bigger models, and even more so with "reasoning models". Regardless, at the scale of 100M users, big one-off costs start looking small. permalink fedilink source parent hideshow 2 child comments replies: [–] adoxographer@lemmy.world 1 point 2 years ago (1 child) But I’d imagine any Chinese operator will handle scale much better? Or? permalink fedilink source parent hideshow 2 child comments replies: [–] orange@communick.news 1 point 2 years ago Maybe? Depends on what costs dominate operations. I imagine Chinese electricity is cheap but building new data centres is likely much cheaper % wise than countries like the US. permalink fedilink source parent
[–] alvvayson@lemmy.dbzer0.com 5 points 2 years ago (2 children) True, but training is one-off. And as you say, a factor 100x less costs with this new model. Therefore NVidia just saw 99% of their expected future demand for AI chips evaporate Even if they are lying and used more compute, it's obvious they managed to train it without access to the large amounts of the highest end chips due to export controls. Conservatively, I think NVidia is definitely going to have to scale down by 50% and they will have to reduce prices by a lot, too, since VC and government billions will no longer be available to their customers. permalink fedilink source parent hideshow 4 child comments replies: [–] bestboyfriendintheworld@sh.itjust.works 3 points 2 years ago (1 child) True, but training is one-off. And as you say, a factor 100x less costs with this new model. Therefore NVidia just saw 99% of their expected future demand for AI chips evaporate It might also lead to 100x more power to train new models. permalink fedilink source parent hideshow 2 child comments replies: [+] ArchRecord@lemm.ee 1 point 2 years ago* (last edited 1 year ago) (1 child) [deleted] permalink fedilink source parent hideshow 2 child comments replies: [–] bestboyfriendintheworld@sh.itjust.works 2 points 2 years ago Thank you. Sounds like good news. permalink fedilink source parent [–] adoxographer@lemmy.world 3 points 2 years ago (1 child) I’m not sure. That’s a very static view of the context. While china has an AI advantage due to wider adoption, less constraints and overall bigger market, the US has higher tech, and more funds. OpenAI, Anthropic, MS and especially X will all be getting massive amounts of backing and will reverse engineer and adopt whatever advantages R1 had. Which while there are some it’s still not a full spectrum competitor. I see the is as a small correction that the big players will take advantage of to buy stock, and then pump it with state funds, furthering the gap and ignoring the Chinese advances. Regardless, Nvidia always wins. They sell the best shovels. In any scenario the world at large still doesn’t have their Nvidia cluster, think Africa, Oceania, South America, Europe, SEA who doesn’t necessarily align with Chinese interests, India. Plenty to go around. permalink fedilink source parent hideshow 2 child comments replies: [–] alvvayson@lemmy.dbzer0.com 2 points 2 years ago (1 child) Extra funds are only useful if they can provide a competitive advantage. Otherwise those investments will not have a positive ROI. The case until now was built on the premise that US tech was years ahead and that AI had a strong moat due to high computer requirements for AI. We now know that that isn't true. If high compute enables a significant improvement in AI, then that old case could become true again. But the prospects of such a reality happening and staying just got a big hit. I think we are in for a dot-com type bubble burst, but it will take a few weeks to see if that's gonna happen or not. permalink fedilink source parent hideshow 2 child comments replies: [–] adoxographer@lemmy.world 1 point 2 years ago Maybe, but there is incentive to not let that happen, and I wouldn’t be surprised if “they” have unpublished tech that will be rushed out. The ROI doesn’t matter, it wasn’t there yet it’s the potential for it. The Chinese AIs are also not there yet. The proposition is to reduce FTEs, regardless of cost, as long as cost is less. While I see OpenAi and mostly startups and VC reliant companies taking a hit, Nvidia itself as the shovel maker will remain strong. permalink fedilink source parent
[–] bestboyfriendintheworld@sh.itjust.works 3 points 2 years ago (1 child) True, but training is one-off. And as you say, a factor 100x less costs with this new model. Therefore NVidia just saw 99% of their expected future demand for AI chips evaporate It might also lead to 100x more power to train new models. permalink fedilink source parent hideshow 2 child comments replies: [+] ArchRecord@lemm.ee 1 point 2 years ago* (last edited 1 year ago) (1 child) [deleted] permalink fedilink source parent hideshow 2 child comments replies: [–] bestboyfriendintheworld@sh.itjust.works 2 points 2 years ago Thank you. Sounds like good news. permalink fedilink source parent
[+] ArchRecord@lemm.ee 1 point 2 years ago* (last edited 1 year ago) (1 child) [deleted] permalink fedilink source parent hideshow 2 child comments replies: [–] bestboyfriendintheworld@sh.itjust.works 2 points 2 years ago Thank you. Sounds like good news. permalink fedilink source parent
[–] bestboyfriendintheworld@sh.itjust.works 2 points 2 years ago Thank you. Sounds like good news. permalink fedilink source parent
[–] adoxographer@lemmy.world 3 points 2 years ago (1 child) I’m not sure. That’s a very static view of the context. While china has an AI advantage due to wider adoption, less constraints and overall bigger market, the US has higher tech, and more funds. OpenAI, Anthropic, MS and especially X will all be getting massive amounts of backing and will reverse engineer and adopt whatever advantages R1 had. Which while there are some it’s still not a full spectrum competitor. I see the is as a small correction that the big players will take advantage of to buy stock, and then pump it with state funds, furthering the gap and ignoring the Chinese advances. Regardless, Nvidia always wins. They sell the best shovels. In any scenario the world at large still doesn’t have their Nvidia cluster, think Africa, Oceania, South America, Europe, SEA who doesn’t necessarily align with Chinese interests, India. Plenty to go around. permalink fedilink source parent hideshow 2 child comments replies: [–] alvvayson@lemmy.dbzer0.com 2 points 2 years ago (1 child) Extra funds are only useful if they can provide a competitive advantage. Otherwise those investments will not have a positive ROI. The case until now was built on the premise that US tech was years ahead and that AI had a strong moat due to high computer requirements for AI. We now know that that isn't true. If high compute enables a significant improvement in AI, then that old case could become true again. But the prospects of such a reality happening and staying just got a big hit. I think we are in for a dot-com type bubble burst, but it will take a few weeks to see if that's gonna happen or not. permalink fedilink source parent hideshow 2 child comments replies: [–] adoxographer@lemmy.world 1 point 2 years ago Maybe, but there is incentive to not let that happen, and I wouldn’t be surprised if “they” have unpublished tech that will be rushed out. The ROI doesn’t matter, it wasn’t there yet it’s the potential for it. The Chinese AIs are also not there yet. The proposition is to reduce FTEs, regardless of cost, as long as cost is less. While I see OpenAi and mostly startups and VC reliant companies taking a hit, Nvidia itself as the shovel maker will remain strong. permalink fedilink source parent
[–] alvvayson@lemmy.dbzer0.com 2 points 2 years ago (1 child) Extra funds are only useful if they can provide a competitive advantage. Otherwise those investments will not have a positive ROI. The case until now was built on the premise that US tech was years ahead and that AI had a strong moat due to high computer requirements for AI. We now know that that isn't true. If high compute enables a significant improvement in AI, then that old case could become true again. But the prospects of such a reality happening and staying just got a big hit. I think we are in for a dot-com type bubble burst, but it will take a few weeks to see if that's gonna happen or not. permalink fedilink source parent hideshow 2 child comments replies: [–] adoxographer@lemmy.world 1 point 2 years ago Maybe, but there is incentive to not let that happen, and I wouldn’t be surprised if “they” have unpublished tech that will be rushed out. The ROI doesn’t matter, it wasn’t there yet it’s the potential for it. The Chinese AIs are also not there yet. The proposition is to reduce FTEs, regardless of cost, as long as cost is less. While I see OpenAi and mostly startups and VC reliant companies taking a hit, Nvidia itself as the shovel maker will remain strong. permalink fedilink source parent
[–] adoxographer@lemmy.world 1 point 2 years ago Maybe, but there is incentive to not let that happen, and I wouldn’t be surprised if “they” have unpublished tech that will be rushed out. The ROI doesn’t matter, it wasn’t there yet it’s the potential for it. The Chinese AIs are also not there yet. The proposition is to reduce FTEs, regardless of cost, as long as cost is less. While I see OpenAi and mostly startups and VC reliant companies taking a hit, Nvidia itself as the shovel maker will remain strong. permalink fedilink source parent
[–] vrighter@discuss.tchncs.de 2 points 2 years ago if, on a modern gaming pc, you can get breakneck speeds of 5 tokens per second, then actually inference is quite energy intensive too. 5 per second of anything is very slow permalink fedilink source parent
[–] orange@communick.news 2 points 2 years ago (1 child) That's becoming less true. The cost of inference has been rising with bigger models, and even more so with "reasoning models". Regardless, at the scale of 100M users, big one-off costs start looking small. permalink fedilink source parent hideshow 2 child comments replies: [–] adoxographer@lemmy.world 1 point 2 years ago (1 child) But I’d imagine any Chinese operator will handle scale much better? Or? permalink fedilink source parent hideshow 2 child comments replies: [–] orange@communick.news 1 point 2 years ago Maybe? Depends on what costs dominate operations. I imagine Chinese electricity is cheap but building new data centres is likely much cheaper % wise than countries like the US. permalink fedilink source parent
[–] adoxographer@lemmy.world 1 point 2 years ago (1 child) But I’d imagine any Chinese operator will handle scale much better? Or? permalink fedilink source parent hideshow 2 child comments replies: [–] orange@communick.news 1 point 2 years ago Maybe? Depends on what costs dominate operations. I imagine Chinese electricity is cheap but building new data centres is likely much cheaper % wise than countries like the US. permalink fedilink source parent
[–] orange@communick.news 1 point 2 years ago Maybe? Depends on what costs dominate operations. I imagine Chinese electricity is cheap but building new data centres is likely much cheaper % wise than countries like the US. permalink fedilink source parent
[–] alvvayson@lemmy.dbzer0.com 3 points 2 years ago (1 child) Wth?! Like seriously. I assume they are running the smallest version of the model? Still, very impressive. permalink fedilink source parent hideshow 2 child comments replies: [–] vrighter@discuss.tchncs.de 3 points 2 years ago i can also run it on my old pentium from 3 decades ago. I'd have to swap 4MiB of weights in and out constantly, it will be very very slow, but it will work. permalink fedilink source parent
[–] vrighter@discuss.tchncs.de 3 points 2 years ago i can also run it on my old pentium from 3 decades ago. I'd have to swap 4MiB of weights in and out constantly, it will be very very slow, but it will work. permalink fedilink source parent
[–] GenosseFlosse@feddit.org 0 points 2 years ago (1 child) Sure you can run it on low end hardware, but how does the performance (response time for a given prompt) compare to the other models, either local or as a service? permalink fedilink source parent hideshow 2 child comments replies: [+] ArchRecord@lemm.ee 1 point 2 years ago* (last edited 1 year ago) [deleted] permalink fedilink source parent
[+] ArchRecord@lemm.ee 1 point 2 years ago* (last edited 1 year ago) [deleted] permalink fedilink source parent