Sam Altman:

A lot of people want local inference, and somehow they assume that the problem is solved now because we have open-source models, we're going to be able to run them on our computers, and so we're not going to need all the stuff in the data center.

But from a global allocation perspective, actually it's quite terrible to have everybody, like, have a bunch of the world's limited supply of DRAM sitting in their own computer and being used five percent of the time.

Like you want maximum utilisation in the cloud if you're trying to deal with this very supply-constrained environment,

And somehow many people's intuition is the opposite of that

TL;DR Guy selling product: "Here's why owning your own product instead of renting it from me is a terrible idea"

all 39 comments

sorted by: hot top controversial new old
[–] 28 points 2 hours ago* (last edited 53 minutes ago) (2 children)

A lot of people want to be billionaires, and somehow they assume the problem is solved because we have unfettered predatory late stage capitalism and oligarchy in America, we're going to be able to amass as much money as we want in our bank accounts, and so we're all going to be rich without oversight, accountability or government interference.

But from a global allocation perspective, actually it's quite terrible to have everybody, like, have a bunch of the world's wealth sitting in a few private bank accounts and being used .05% of the time.

Like you want maximum circulation of currency if you're trying to reduce inequality in this artificial scarcity environment,

And somehow, Americans' intuitions are the opposite of that.

FTFY Sam.

  • source
  • hideshow 2 child comments
  • [–] 6 points 1 hour ago

    yeah my computer with all it's computing power is turned off most of the time

    and so is pretty much everything I own except my phone

  • source
  • [–] 23 points 3 hours ago (2 children)

    No.. he actually is right. A local machine will sit idle most if the time.

    A data center can balance loads globally.

    In terms of resources.. data centers ARE cheaper.

    However ... there are other reasons to use local interference that might be more important than added cost

  • source
  • hideshow 2 child comments
  • [–] 2 points 1 hour ago

    You can aggregate spare LLM capacity in a collaborative p2p fashion. But my reasons would be to use spare PV capacity to run an uncensored model during daytime with full privacy.

  • source
  • parent
  • [–] 5 points 2 hours ago

    Going cheap almost always ends in a more expensive repair bill down the line.

    Likewise, doing the most convenient thing to satisfy one's own desires is probably the most ill-advised course of action. Mainly, because it makes other people take the most convenient course of action to remove the other person from being a problem in their lives altogether.

  • source
  • parent
  • [–] 10 points 2 hours ago

    Sounds like a problem for the sharing economy: kinda like you're able to do https://foldingathome.org/

    I'm sure someone will come up with a solution that lets you rent out excess capacity from your gpu cluster.

  • source
  • [–] 33 points 4 hours ago

    And there it is.

    “We made components scarce, so you have no choice but to use our cloud services where we control everything and you rent the privilege from us.”

    Literally called it. Fuck these assholes with everything rusty.

  • source
  • [–] 20 points 4 hours ago (1 child)

    I bet he owns a car that gets less than 5% usage.

  • source
  • hideshow 1 child comment
  • [+] 1 point 1 hour ago
    [–] 75 points 6 hours ago (1 child)

    Funny how there weren't any systemic supply issues until someone decided to buy up all the RAM ...

  • source
  • hideshow 1 child comment
  • [–] 42 points 6 hours ago (3 children)

    Ram is so cheap to make that until the AI demand companies basically had to create new levels of density/speed and sell for less just to keep the business afloat. This is like when cellphone companies tried to claim megabytes were a precious finite resource. We live in a post scarcity society and these billion assholes are like “well that just won’t do”.

    It reminds me of that kid we all know who just learned economics from school house rock and then goes “now if I can simply find every evee Pokémon card in the world and burn them all except for the one I have, I’ll be rich!”

    It’s just so frustrating to see this sheer stupid amount of greed all of the time. Thanks for coming to my TEDex talk.

  • source
  • hideshow 3 child comments
  • [–] 45 points 6 hours ago (2 children)

    Personal computers are a ridiculous idea, you should use our scalable mainframes instead --IBM in the 80s, probably

  • source
  • hideshow 2 child comments
  • [–] 97 points 7 hours ago (1 child)

    McDonald’s can probably make hamburgers more efficiently than everyone can at home, but I’m not eating that shit any more than I’m giving Sam Altman my money or data.

  • source
  • hideshow 1 child comment
  • [–] 13 points 5 hours ago

    Exclusively pursuing efficiency manifests as intrinsically brittle systems and monocultures. Maximum utilisation is only desirable for strip mining.

    Disperse, choreographed capabilities provides reliability and resiliency, slack permits flexibility and predictable maintenance.

    All this is orthogonal to the legitimacy of the use and demand of the grift this clown is selling.

  • source
  • [–] 41 points 7 hours ago (1 child)

    Some of you will have to sacrifice for the greater good, but it won’t be him.

  • source
  • hideshow 1 child comment
  • [–] 11 points 6 hours ago

    As if the ONLY thing your memory could be used for is running AI models.

    Hey buddy, we need this shit to use our BROWSERS or to play GAMES. I have a GPU in my computer that can run most new AI models when quantized at a speed that would be perfectly reasonable for any home user that doesn't want it to literally replace every ounce of their thinking, and I bought that shit before ChatGPT was even a thing, because I wanted to play fucking video games.

    I have a NAS with 16 gigs of RAM and a consumer-grade CPU that's a few generations old, and it can still run models with zero GPU or AI accelerator if you give it a little time.

    Is it going to be perfectly tuned, and run at the highest possible speed if you design a datacenter just for AI models? Of course it is. But do most people on this planet need that or give a fuck about it? Absolutely not.

  • source
  • [–] 27 points 7 hours ago (1 child)

    Can you imagine everyone carrying around gigabytes of memory, in their pocket? Unthinkable.

  • source
  • hideshow 1 child comment
  • [–] 24 points 7 hours ago (2 children)

    Central planning but by the private sector

  • source
  • hideshow 2 child comments
  • [–] 12 points 6 hours ago (1 child)

    On a somewhat related note, the private sector routinely does central planning in areas dominated by few large players. E.g. Walmart. They use a version of the mathematical models the Soviets came up with to do central planning at Gosplan.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 10 points 7 hours ago (4 children)
  • [–] 9 points 7 hours ago

    See this is why, for most agentic work, I use inference providers that clearly run Open Weights models and have it in their terms that they won’t use my data to train models. Then, for fucking around, or for more sensitive work or my PII stuff, I use local models.

  • source
  • [+] 1 point 7 hours ago