this post was submitted on 02 Jul 2024
545 points (100.0% liked)

TechTakes

1489 readers
33 users here now

Big brain tech dude got yet another clueless take over at HackerNews etc? Here's the place to vent. Orange site, VC foolishness, all welcome.

This is not debate club. Unless it’s amusing debate.

For actually-good tech, you want our NotAwfulTech community

founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
[–] [email protected] 118 points 5 months ago* (last edited 5 months ago) (22 children)

there’s this type of reply guy on fedi lately who does the “well actually querying LLMs only happens in bursts and training is much more efficient than you’d think and nvidia says their gpus are energy-efficient” thing whenever the topic comes up

and meanwhile a bunch of major companies have violated their climate pledges and say it’s due to AI, they’re planning power plants specifically for data centers expanded for the push into AI, and large GPUs are notoriously the part of a computer that consumes the most power and emits a ton of heat (which notoriously has to be cooled in a way that wastes and pollutes a fuckton of clean water)

but the companies don’t publish smoking gun energy usage statistics on LLMs and generative AI specifically so who can say

[–] [email protected] 61 points 5 months ago (3 children)

I joined the Microsoft climate solutions team, I think doing so has crushed all my hopes. That team is doing small things like "let's all go plastic free for July!" Top 20 company in the world and the best we can do is go plastic free for a month? Meanwhile MSFT is not on course to meet their own climate goal by 2030

[–] [email protected] 34 points 5 months ago (2 children)

we're aiming to do 1% more recycling this year 💪💪💪

[–] [email protected] 16 points 5 months ago (1 children)

recycling plastic is not as shiny as advertised, just burn this shit for energy instead

[–] [email protected] 21 points 5 months ago (2 children)

I like it. Burn trash to power data centers running LLMs. Garage in, garbage out

[–] [email protected] 17 points 5 months ago

it's better used for municipal heating, you know, something actually useful

[–] [email protected] 8 points 5 months ago

Instructions unclear, burned datacenter instead.

(To any 3 letter agencies reading this, I'm making a joke, I'm not pro burning down datacenters, that is Yuds thing).

[–] [email protected] 7 points 5 months ago (1 children)

Less recycling more reusing bottles for molotovs please

[–] [email protected] 10 points 5 months ago

reduce, reuse, recycle, retaliate

[–] [email protected] 7 points 5 months ago

I'm on a group text with your exec colleagues. Disheartening would be an understatement.

load more comments (1 replies)
[–] [email protected] 49 points 5 months ago (1 children)

“It only uses 5x as much energy as a regular search! Think of how much energy YOU’RE using with searches!” Okay, so you’re just using 5x as much energy for worse results? And also probably doing it more often than people who just use a normal search engine, because they don’t expect the search engine to talk to them. I’ve never understood how that was supposed to be an exoneration for it, even without taking into account that nobody ever seems to know whether or not that figure includes energy spent on training.

[–] [email protected] 48 points 5 months ago

AI bros use literally the same whatabout excuses for their ghastly power consumption that I know from years of bitcoin bros doing the same

like, at least christmas lights bring joy

[–] [email protected] 17 points 5 months ago (3 children)

there’s this type of reply guy on fedi lately who does the “well actually querying LLMs only happens in bursts and training is much more efficient than you’d think and nvidia says their gpus are energy-efficient” thing whenever the topic comes up

This kind of person (also happened a lot with cryptocurrencies) always goes 'that isn't how it works, this isn't a problem' then doesn't explain what the mistake is you are supposed to have made, and then a few weeks/months/days/search later it is revealed that it was how it works and it is a huge problem. And it is so annoyingly common im very happy with the moderation here.

[–] [email protected] 10 points 5 months ago (1 children)

I just want to be able to legally punch people who think NFTs would bring down TicketMaster. It's the peak of not understanding how things work and injecting a solution just because it's high tech.

[–] [email protected] 7 points 5 months ago

one that already exists - you can go to a show and your "ticket" is a hash you were sent in email and they cross it off at the door

[–] [email protected] 9 points 5 months ago (1 children)

it’s so common that I wish there was a specific name to put to that and every other type of reply guy that cryptocurrencies, chan culture, and meme stocks “gifted” us. it feels like the bullshit tactics cropped up faster than anyone really was able to catalog them, though a lot of them are really recognizable when you’ve seen them a few times.

[–] [email protected] 5 points 5 months ago (1 children)

I'm guessing that "obscurantist proponent" is not exactly an ideal option, sorta lacks that punch for precise poignant use in an enraged soliloquy

I'll give it a ponder see if my brain delivers an option

[–] [email protected] 3 points 5 months ago

weak attempt: replyribbon; linear, unsophisticated, prone to reuse without much change, and extremely easy to rip to shreds

[–] [email protected] 8 points 5 months ago

jeez fuck this is just reality negationism at this point

you could snipe it down with canned response like r/buttcoin does (there are only so many types of these people), and it could count as a win because reply guy isn't there to win debate, he's there to spread his propaganda, but here, it's too much effort for no gain because nobody will be swayed anyway

[–] [email protected] 12 points 5 months ago (2 children)

I live in Ireland and these companies keep trying to build data centres here even though they are using way too much of our energy as is :)) From an Irish environmental org:

A few years ago when researching new gas power stations in Ireland, we noticed that many of these developments were being proposed alongside another type of infrastructure: data centres. [...] As of June 2023, there are 82 operational data centres in Ireland, with another 14 under construction. Additionally, 40 data centres have had planning permission approved with another 12 awaiting a decision. [...] Data centres account for 18% of all electricity use in Ireland [...] This is putting unprecedented strain on the electricity grid, with grid operator Eirgrid estimating that data centres may account for up to 27% of Ireland’s electricity demand by 2028. The Commission for Regulation of Utilities (CRU) in Ireland has noted a risk of rolling electricity blackouts...

[–] [email protected] 7 points 5 months ago* (last edited 5 months ago) (1 children)

ah yes, ireland, the land of just-close-enough to LIXP/TH(D|E)/etc but with different-enough property prices and power capabilities to make it very attractive to the discerning dc builder

(there's also some of the tax regs shit but aiui that isn't as gaping a hole as it used to be)

[–] [email protected] 6 points 5 months ago (1 children)

We're actually having a housing crisis at the moment (like everywhere else) so I dunno about property prices...I think it's the extremely receptive attitude of the Irish government.

[–] [email protected] 6 points 5 months ago (1 children)

do you have a housing crisis or do you have a landlord crisis (like how the US has 16 million vacant homes)

[–] [email protected] 6 points 5 months ago

landlord crisis and a vulture fund crisis. don't get me started because I'll be frothing at the mouth in seconds

[–] [email protected] 5 points 5 months ago (1 children)

I thought the Irish were always pretty good at kicking out colonialist efforts. Maybe just try the old reliable approach.

[–] [email protected] 6 points 5 months ago (1 children)

Our entire economy since the 80s has been reliant on neo-colonial relationships with multinational companies unfortunately

[–] [email protected] 5 points 5 months ago (1 children)

that is unfortunate because Ireland was my backup plan for if the US falls (half of my grandparents were citizens)

[–] [email protected] 7 points 5 months ago

Yeah imo Ireland has never had a left-wing government, we tend towards right-populist/right centrist/neoliberal. "Stable but shit" is how I described it to my brother just this hour gone while discussing politics. We're currently seeing the beginnings of an organized racist right as well

[–] [email protected] 5 points 5 months ago

I always thought data centers ran clean and dirty loops of cooling (as far as computers are concerned).
The clean loop has all the chemicals and whatnot to keep cooling blocks and tubing "safe". The dirty side is just plain old water. And a big heat exchanger transfers the heat from the clean (hot) loop to the "dirty" (cold) side.
Is there really that much pollution in that? Can't be worse than rain going through storm drains or whatever.

But AI does use a phenomenal amount of power.
And, IMO, it's a problem compared to the lack of value people are getting from AI.
The new Blackwell B200 consumes 1.2kw of power, and will produce 1.2kw of heat.
A cooling system with a COP of 5 needs to consume 240w to dissipate this.
The backplane for the B200 holds 8 of these GPUs in a 10 RU space, and with overheads will peak 14.3kw (cooling would be 3kw consumption).
So, a 42u data center rack with 3 of these, supporting hardware and UPS efficiencies (80%) is going to be 52kw (+10kw cooling). 62kw total, which is like 4 homes drawing their full load all the time.

I hope they finally find an application for AI, instead of just constantly chasing the dragon with more training, more parameters, more performance etc.

[–] [email protected] 5 points 5 months ago* (last edited 5 months ago) (2 children)

The last part is absolutely false. The Nvidia H100 TDP is like 700W, though ostensibly configurable. The B200 is 1000W. The AMD MI300X is 750W.

They also skimp on RAM with many SKUs so you have to buy the higher clocked ones.

They run in insane power bands just to eek out a tiny bit more performance. If they ran at like a third of their power, I bet they would be at least twice as power efficient, and power use scales over nonlinearly with voltage/clock speed.

But no, just pedal to the metal. Run the silicon as hard as it can, and screw power consumption.

Other AI companies like Cerebras are much better, running at quite sane voltages. Ironically (or perhaps smartly), the Saudis invested in them.

[–] [email protected] 25 points 5 months ago (5 children)

Other AI companies like Cerebras are much better, running at quite sane voltages. Ironically (or perhaps smartly), the Saudis invested in them.

it’s real bizarre you edited this in after getting upvoted by a few people

[–] [email protected] 19 points 5 months ago (1 children)

banned them for the subtle spam attempt

[–] [email protected] 24 points 5 months ago

“it’s just lemmy, my reddit shit will work fine there” says meme stock marketer who’s never been subjected to any kind of scrutiny

load more comments (4 replies)
[–] [email protected] 8 points 5 months ago

wow, exemplary performance

load more comments (16 replies)