784
OpenAI is so cooked and I'm all here for it (lemmy.dbzer0.com)
submitted 2 years ago by to c/techtakes@awful.systems
207 comments fedilink hide all child comments
 
‪Ed Zitron‬ ‪@edzitron.com‬ · 3h OpenAI is completely screwed. If they're losing money on $200/month, they're losing it on $20/month and their real costs are so much higher than they can ever get anyone to pay. A pale horse. - Screenshot from an article: https://techcrunch.com/2025/01/05/openai-is-losing-money-on-its-pricey-chatgpt-pro-plan-ceo-sam-altman-says/
you are viewing a single comment's thread
view the rest of the comments
[–] 23 points 2 years ago (9 children)

Okay, explain. What kinds of low hanging fruit?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 9 child comments
    •
  • [+] -13 points 2 years ago* (last edited 2 years ago) (8 children)

    quants are pretty basic. switching from floats to ints (faster instruction sets) are the well known issues. both those are related to information theory, but there are other things I legally can't mention. shrug. suffice to say the model sizes are going to be decreasing dramatically.

    edit: the first two points require reworking the base infrastructure to support which is why they havent hit widespread adoption. but the research showing that 3 bits is as good as 64 is intuitive once you tie the original inspiration for some of the AI designs. that reduction alone means you can get 21x reduction in model size is pretty solid.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 8 child comments
    •
  • [–] 26 points 2 years ago (7 children)

    both those are related to information theory, but there are other things I legally can’t mention. shrug.

    hahahaha fuck off with this. no, the horseshit you’re fetishizing doesn’t fix LLMs. here’s what quantization gets you:

    • the LLM runs on shittier hardware
    • the LLM works worse too
    • that last one’s kinda bad when the technology already works like shit

    anyway speaking of basic information theory:

    but the research showing that 3 bits is as good as 64 is intuitive once you tie the original inspiration for some of the AI designs.

    lol

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 7 child comments
    •
  • [–] 12 points 2 years ago (3 children)

    I have seen these 3 bit ai papers on hacker news a few times. And the takeaway apparently is: the current models are being pretty shitty at what we want them to do, and we can reach a similar (but slightly worse) level of shittyness with 3 bits.

    But that doesn't say anything about how both technologies could progress in the future. I guess you can compensate for having only three bits to pass between nodes by just having more nodes. But that doesn't really seem helpful, neither for storage nor compute.

    Anyways yeah it always strikes me as a kind of trend that maybe has an application in a very specific niche but is likely bullshit if applied to the general case

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 3 child comments
    •
  • [–] 7 points 2 years ago (1 child)

    It's actually super easy to increase the accuracy of LLMs.

    import pytorch # or ollama or however you fucking dorks use this nonsense
from decimal import Decimal

    I left out all the other details because it's pretty intuitive why it works if you understand why floats have precision issues.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •