It's not an answer you'd get from Claude — it's real, organic content:

• 👶written by a genuine human
• 💡delivering original ideas and language
• 🚀going above and beyond to answer
• ✨synergizing cross-platform initiatives

(🤪 this is a joke)

I think the issue is also that you need some serious hardware to get good inference speed when your devs are working, but then most of the time this hardware will be under utilized.

That being said you can get good performance from indie inference farms, at a fraction of the cost of the big US labs. I think it's a great compromise and in a few months the open models will be near parity with opus 4.6 which is really all you need for most tasks.

Bigs definitely do, and anyone with confidential data should be.

Because the people selling the AI wants to make sure their customers don't know about this. It's all about causing a dependency so they get subscription income forever.

Yes, the big proprietary cloud models have an edge, but it is narrow and the open-weight models are constantly closing the gap. There is no moat when it comes to AI models and no company has yet discovered some secret special sauce to improve their model significantly over others.

Running the latest and greatest open-weight GLM, Kimi, or Qwen model is basically equivalent to running the previous latest and greatest version of Claude. So if you were happy with Claude then, you'll basically be happy with an open-weight model now.

Match current Claude is not, but Claude 6-12 months ago should be possible using Open model

Mostly down to frameworks (the bits around the LLM like RAG, memory, prompts, agents etc.) now. The ability to just throw more tokens at the problem is also super important. And you can because you're just paying for electricity (and CapEx for the hardware), not tokens from companies that are doing pre-IPO monetization (i.e. tokens gonna go up, way up). They've been losing money hand over fist to gain market share and pump the idea, that was never going to last.

No that's not how this works. Inference is cheap and efficient. AI companies are bankrupting themselves with training costs that they need to recoup back by selling inference. Open-weight models have already been trained.

Also, going big in terms of model size shows diminishing marginal returns on accuracy, not efficiency of scale. Smaller models are way more efficient and consistently catch up to the largest models, which is why today's SOTA 27 billion parameter model competes with yesterday's SOTA 500+ billion parameter model.

There's a big difference between training a model, running a model, and running a model at scale.

A small, self hosted setup will have lower accuracy and queries per second, and it will have a cost, but the cost will be no more than playing a videogame. You'll still have something surprisingly accurate and responsive for some tasks, like being a wiki interface or something.

Remember that some of these models can run on a standard smartphone, and all the hoopla when people found that chrome was downloading models onto people's devices.

At least there they have hard numbers, without a CEO dreaming about future possibilities and whatnot.

Hence asking questions

a 128GB framework desktop could do that job. it's increased a bit in price since i last looked at it but €4500 isn't that much for a company.