76
How does AI use so much power?
(lemmy.world)
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
this post was submitted on 10 Jul 2025
76 points (96.3% liked)
No Stupid Questions
42196 readers
479 users here now
No such thing. Ask away!
!nostupidquestions is a community dedicated to being helpful and answering each others' questions on various topics.
The rules for posting and commenting, besides the rules defined here for lemmy.world, are as follows:
Rules (interactive)
Rule 1- All posts must be legitimate questions. All post titles must include a question.
All posts must be legitimate questions, and all post titles must include a question. Questions that are joke or trolling questions, memes, song lyrics as title, etc. are not allowed here. See Rule 6 for all exceptions.
Rule 2- Your question subject cannot be illegal or NSFW material.
Your question subject cannot be illegal or NSFW material. You will be warned first, banned second.
Rule 3- Do not seek mental, medical and professional help here.
Do not seek mental, medical and professional help here. Breaking this rule will not get you or your post removed, but it will put you at risk, and possibly in danger.
Rule 4- No self promotion or upvote-farming of any kind.
That's it.
Rule 5- No baiting or sealioning or promoting an agenda.
Questions which, instead of being of an innocuous nature, are specifically intended (based on reports and in the opinion of our crack moderation team) to bait users into ideological wars on charged political topics will be removed and the authors warned - or banned - depending on severity.
Rule 6- Regarding META posts and joke questions.
Provided it is about the community itself, you may post non-question posts using the [META] tag on your post title.
On fridays, you are allowed to post meme and troll questions, on the condition that it's in text format only, and conforms with our other rules. These posts MUST include the [NSQ Friday] tag in their title.
If you post a serious question on friday and are looking only for legitimate answers, then please include the [Serious] tag on your post. Irrelevant replies will then be removed by moderators.
Rule 7- You can't intentionally annoy, mock, or harass other members.
If you intentionally annoy, mock, harass, or discriminate against any individual member, you will be removed.
Likewise, if you are a member, sympathiser or a resemblant of a movement that is known to largely hate, mock, discriminate against, and/or want to take lives of a group of people, and you were provably vocal about your hate, then you will be banned on sight.
Rule 8- All comments should try to stay relevant to their parent content.
Rule 9- Reposts from other platforms are not allowed.
Let everyone have their own content.
Rule 10- Majority of bots aren't allowed to participate here. This includes using AI responses and summaries.
Credits
Our breathtaking icon was bestowed upon us by @Cevilia!
The greatest banner of all time: by @TheOneWithTheHair!
founded 2 years ago
MODERATORS
It's mostly the training/machine learning that is power hungry.
AI is essentially a giant equation that is generated via machine learning. You give it a prompt with an expected answer, it gets run through the equation, and you get an output. That output gets an error score based on how far it is from the expected answer. The variables of the equation are then modified so that the prompt will lead to a better output (one with a lower error).
The issue is that current AI models have billions of variables and will be trained on billions of prompts. Each variable will be tuned based on each prompt. That's billions to the power of billions of calculations. It takes a while. AI researchers are of course looking for ways to speed up this process, but so far it's mostly come down to dividing up these billions of calculations over millions of computers. Powering millions of computers is where the energy costs come from.
Unless AI models can be trained in a way that doesn't require running a billion squared calculations, they're only going to get more power hungry.
This is a pretty great explanation/simplification.
I'll add that because the calculations rely on floating point math in many cases, graphics chips do most of the heavy processing since they were already designed for this pipeline in mind with video games.
That means there's a lot of power hungry graphics chips running in these data centers. It's also why NVidia stock is so insane.
It's kinda interesting how the most power-consuming uses of graphics chips — crypto and AI/ML — have nothing to do with graphics.
(Except for AI-generated graphics, I suppose)
Thinking of a modern GPU as a "graphics processor" is a bit misleading. GPUs haven't been purely graphics processors for 15 years or so, they've morphed into general-purpose parallel compute processors with a few graphics-specific things implemented in hardware as separate components (e.g. rasterization, fragment blending).
Those hardware stages generally take so little time compared to the rest of the graphics pipeline that it normally makes the most sense to have far more silicon dedicated to general-purpose shader cores than the fixed-function graphics hardware. A single rasterizer unit might be able to produce up to 16 shader threads worth of fragments per cycle, so even if your fragment shader is very simple and only takes 8 cycles per pixel, you can keep 8x16 cores busy with only one rasterizer in this example.
The result is that GPUs are basically just a chip packed full of a staggering number of fully programmable floating-point and integer ALUs, with only a little bit of fixed hardware dedicated to graphics squeezed in between. Any application which doesn't need the graphics stuff and just wants to run a program on thousands of threads in parallel can simply ignore the graphics hardware and stick to the programmable shader cores, and still be able to leverage nearly all of the chip's computational power. Heck, a growing number of games are bypassing the fixed-function hardware for some parts of rendering (e.g. compositing with compute shaders instead of drawing screen-sized rectangles, etc.) because it's faster to simply start a bunch of threads and read+write a bunch of pixels in software.
Would AI inferencing, or training be better suited to a quantum computer? I recall thouse not being great at conventional math, but massively accelerates computations that sounded similar to machine learning.
My understanding of quantum computers is that they're great a brute forcing stuff, but machine learning is just a lot of calculations, not brute forcing.
If you want to know the square root of 25, you don't need to brute force it. There's a direct way to calculate the answer and traditional computers can do it just fine. It's still going to take a long time if you need to calculate the square root of a billion numbers.
That's basically machine learning. The individual calculations aren't difficult, there's just a lot to calculate. However, if you have 2 computers doing the calculations, it'll take half the time. It'll take even less time if you fill a data center with a cluster of 100,000 GPUs.