[–] 2 points 14 hours ago

Though bear in mind that it doesn't insert new information into the model that it didn't have before, it just modifies how the model responds given the information that it already knows.

That's generally a bit of a quibble since "censored" models were still generally trained with the information they're meant to refuse to give - otherwise they wouldn't understand what they're supposed to refuse to do. But worth mentioning in case a situation arises like with AI image models, where some models were trained using image that literally didn't include any genitals in them and as a result no amount of de-censorship is going to help them make crotches that don't look like they were designed by Cronenberg.

  • source
  • parent
  • context
  • [–] 1 point 1 day ago

    Depends on the framework you're using. I'm using Qwen Code Companion (via a VS Code IDE plugin) and it's got some kind of tiered system built into it for adding tool definitions to context, it's got a standard built-in set that's always mentioned but other stuff only gets added to the context under certain circumstances. I can't recall all the details, I found out about it when I was chatting with Qwen Code to ask it why it hadn't used a particular MCP I'd given it to solve a problem despite it seeming like a useful tool for the task and we got into a discussion about to configure it. As I recall it's something along the lines of most tools only being present in context by default as little one-liner stubs and the agent has to tell the framework "that one looks like it might be useful for what I'm doing" for the full definition to get loaded.

    Most of the time I'm working on small scripts and specialized applications these days, and codebase memory is overkill for those so I haven't actually seen it in use much.

  • source
  • parent
  • context
  • [–] 8 points 1 day ago

    I switched to LM Studio as my generic "run whatever GGUF I've come across today" platform and I quite like it. It doesn't have the weird setup and configuration thing Ollama does, it's got a nice built-in chat system like Kobold does, and it does dynamic model loading on demand so my various random scripts can call into it for whatever they need without me having to do any manual work.

    Running stuff with straight llama would probably be more computationally efficient, but I value easy interfaces too.

  • source
  • parent
  • context
  • view more: next ›