you are viewing a single comment's thread
view the rest of the comments
[–] 3 points 3 months ago

Yup, you want memory accessible to the GPU for local AI. AMD Strix Point and Mac devices are popular options. CPU can run LLMs but very slowly. I've got 32 GB of RAM and 8 VRAM and it's borderline useless for models that don't fit in the VRAM.

  • source
  • parent