75
Why your local LLM feels dumber than it is (forum.level1techs.com)
submitted 4 days ago* by to c/technology@lemmy.world
103 comments fedilink hide all child comments
you are viewing a single comment's thread
view the rest of the comments
[–] 3 points 3 days ago (1 child)

Yeah, I'm just saying it's a really easy way to get started, without being overwhelmed by formats and quantizations and top-k and stuff. You run it, it goes, then you can move to something better.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 2 points 3 days ago*

    Its sleazy software, and IMO its defaults provide a really terrible 1st time experience. It convinces 1st time users that locally run LLMs are terrible, and goes out of its way to be incompatible with other stuff.

    I think LM Studio would be a much better "1st time" recommendation. Its not perfect either, but its lightyears better, not an active con like ollama, and its smoother to "graduate" from it to croco.cpp or llama.cpp or something.

  • permalink
  • fedilink
  • source
  • parent