1166
What AI art reveals (thelemmy.club)
submitted 3 days ago by to c/memes@sopuli.xyz
187 comments fedilink hide all child comments
 
you are viewing a single comment's thread
view the rest of the comments
[–] 1 point 1 day ago (2 children)

You can't fix it with a pure LLM, no. But very few of the major models are pure LLMs anymore. They've got LLMs built in to them as subsystems, but they've got control systems layered on top of those to adjust their behaviors. And those don't have to be trained purely with next word prediction.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 13 hours ago (1 child)

    The problem with that is that, it seems like it will become an unwieldy piece of spaghetti code, even assuming the code wasn't written by LLMs.

    An LLM is a crazily complex thing but at least it's text in, text out, one input, one output.

    As soon as you start bolting more and more things onto it to deal with special conditions, you have to maintain all those things, and understand how they interact.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 1 point 10 hours ago

    I'm not talking about bolting things together at random. I'm talking about things like mixture of experts or chain of reasoning, which apply structures to coax more rational behavior out of the systems. You do have to generate new bodies of training data for them, but there are ways of doing that algorithmically. This allows them to move away from the strict "regurgitate the Internet" models of the previous versions. There isn't, currently, this kind of structure to drive the systems towards an honest assessment of their own knowledge and limits, but it's basically inevitable that such a system will be developed, because it's a valuable kind of behavior to be able to favor.

  • permalink
  • fedilink
  • source
  • parent