182
Thunderbird Team Unveils Thunderbolt Self-Hostable AI Client (linuxiac.com)
submitted 4 months ago by to c/technology@lemmy.world
77 comments fedilink hide all child comments
 

I am tired boss...

you are viewing a single comment's thread
view the rest of the comments
[–] 26 points 4 months ago (6 children)

Sounds like this is in the same space as OpenWebUI? It would be good to have some more choice there.

  • permalink
  • fedilink
  • source
  • hideshow 6 child comments
    •
  • [–] 22 points 4 months ago* (last edited 4 months ago) (3 children)

    It sounds like a step further than open-webui; it’s an enterprise grade client-server model for access to agents, workflows, and centralized knowledge repositories for RAG.

    In addition to local chatbot for executive/admin use, I can see this being the backend for developers running Cursor or some other AI enhanced IDE, with local knowledge stores holding proprietary documents and running against local large models.

    I am also curious about time share and prioritization of resources; I assume it would queue simultaneous requests. Presumably this would let you more effectively pool local compute, rather than providing A100 GPUs to each developer that may sit unused when they’re not working.

    Edit: Somewhat impressively, this whole stack does not even include a local inference provider; so it does everything except local models right now, and requests are forwarded to cloud inference providers (Anthropic, OpenAI, etc). But it does have the backend started for rate limiting and queuing, and true “fully offline/local” is on the roadmap, just not there yet.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 3 child comments
    •
  • [–] 8 points 4 months ago* (last edited 4 months ago) (1 child)

    After reading through the GitHub docs, the most impressive thing is that they open sourced their Thunderbolt coding agent for Claude Code. There are quite a few skills available for implementation planning, dependency/build environment setup, coding, linting/cleanup, QA, and managing agent pull requests. Pretty good examples if you are looking at building Claude Code skills.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 1 point 4 months ago*

    Where are you seeing that it's useable for RAG? I've gone through the github and not seeing anyting very specific that way.

    Edit: good lord their documentation is shit. Spend a few tokens on a proper mkdocs site or something.

  • permalink
  • fedilink
  • source
  • parent
  • [–] 21 points 4 months ago

    At last a useful comment thread about the actual functionality in question.

    While I am not moving back to reddit, Threadiverse is just terrible on any nuanced conversations on modern ML tools and approaches.

    The tech is not the issue here. It has legitimate use cases and it is here to stay (this is not a blockchain pump and dump scheme ala Web 3.0). The issue here are American tech companies and broader support for crime/corruption in US society (as of today, doesn't mean that this can't/won't change in 20-30 years). We need truly independent open source systems and tools.

    I am aware OpenWebUI is based in San Francisco and Mozilla Foundation is based in the US. I am always on the lookout for alternatives.

  • permalink
  • fedilink
  • source
  • parent
  • [–] 4 points 4 months ago

    It looks like it, it doesn't seem to be doing any self-referencing/agentic things out of the box so the end user would need to build a bit to cover their specific use cases.

    They seem to be aiming more at the startup/small company demographic than at self-hosters. This is just based on a skim of the repo and their product page, I haven't looked at it too hard yet.

  • permalink
  • fedilink
  • source
  • parent