you are viewing a single comment's thread
view the rest of the comments
[–] 1 point 3 days ago (1 child)

Bitcoin went through a similar trajectory. Graphic cards to specualty chips.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 1 point 21 hours ago (1 child)

    AI won't really do that, because it's a very different workload.

    Mining Bitcoin needs very little memory but extremely complex maths calculations. It has a built-in difficulty mechanism that makes the math more difficult if mining goes too fast.

    ASICs are unbeatable when it comes to super-fast pure calculation.

    AI work is mostly memory-bound. You need huge amounts of RAM (24GB for a somewhat decent model, a few TB for frontier models) and then you perform pretty simple vector math on that huge amount of data in memory. So the main limiting factors are RAM size and bandwidth. The computation power of a GPU doesn't matter that much. Also GPUs contain ASICs for vector math already.

    Standalone ASICs have no advantage over GPUs for this workload. The only thing you could do with a specialty chip is remove some unnecessary parts of a GPU to save a bit of cost and energy, thus turning a GPU that can be repurposed or resold if the AI bubble pops into a single-purpose device that has to be discarded if the bubble pops.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 1 point 19 hours ago (1 child)

    You bring up some good points. Just FYI my research was specifically on ai, but a very specific branch in college. So not llm but only somewhat llm flavored.

    The are already including ai chips in consumer hardware. And your thinking of llm specific limitations. But the smaller models can absolutly be thrown into hardware. Its just the algorithms are moving so fast that the hardware needs to be flexible enough. Thars the biggest reason we dont see more hardware faster than gpus. Hooe that makes sense!

  • source
  • parent
  • hideshow 2 child comments
  • [–] 1 point 2 hours ago

    I was gonna bring up changing algorithms too, but didn't because the comment got too big already.

    NPUs are just stripped-down GPUs with less flexible instruction set. They don't meaningfully advance performance over a GPU, but instead reduce cost.

  • source
  • parent