1295
days of future past (thelemmy.club)
submitted 3 months ago by to c/fuck_ai@lemmy.world
152 comments fedilink hide all child comments
 

sauce

you are viewing a single comment's thread
view the rest of the comments
[–] 5 points 3 months ago (4 children)

That and most of the cost of the data centers revolve around the servers and support for those servers ie. Cooling, power racks ect. I've heard it can be cheaper to build a whole new building than bringing one, even currently operating one, up to new specs.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 4 points 3 months ago (2 children)

    This is also extremely true!

    Those AI servers are probably a discounted cost of around 3-5k per U (I'm probably low in this estimate but I have a really hard time believing they're actually paying $20k+ per GPU), probably about 40U per rack is actually loaded with servers, so if we round the footprint of a server rack to 4 square feet (because I don't feel like actually calculating it out right now) that's about $30,000-50,000 per square foot.

    I have a hard time imagining any kind of structure costing anywhere near 1 order of magnitude less than the cost per square foot of the servers

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 5 points 3 months ago (1 child)

    Those AI servers are probably a discounted cost of around 3-5k per U (I'm probably low in this estimate but I have a really hard time believing they're actually paying $20k+ per GPU),

    They're arranging 72 Blackwell GPUs into each server rack, at around a price of $3 million, in a cabinet that is 2236mm x 600 mm x 1068mm. That's approximately a 7 square foot footprint, so about $430,000 per square foot of server. There's obviously a need for spacing between servers, but you're basically underestimating by an order of magnitude.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 1 point 3 months ago

    Yeah my assumption is that the actual price paid for compute is a lot lower than what Nvidia puts in their press releases. It's like Dell telling you your bog stand R760 will cost $30k on their website then you talk a rep and they mark it down to 5-10k in the first round of negotiations and if you buy enough of them they'll knock a couple thousand more off the price.

    I did however forget to factor in the cost of the server, but they're so often just Supermicro white boxes in these datacenters and I'm just throwing out vague numbers based purely on my knowledge of roughly what things generally cost normally

  • permalink
  • fedilink
  • source
  • parent