you are viewing a single comment's thread
view the rest of the comments
[–] 10 points 16 hours ago (1 child)

I do hope this is a sign that AI will follow the same trend that computing in general did, where mainframes eventually gave way to desktop PCs and local computing. Certainly, there will always be tasks that would require a frontier model on a datacenter to complete, but I suspect an increasing share would be done with smaller local models, especially given that security and control are all significant concerns for many people and organizations.

  • source
  • hideshow 2 child comments
  • [–] [S] 10 points 16 hours ago

    It's definitely happening, Qwen 3.8 is amazing at solving fairly complex tasks at this point, and it's straight up better than frontier models from like half a year ago. So, if we get another jump in local models that gets them to the capability of the current frontier, that's good enough for the vast majority of tasks most people do.

    There are also tricks like https://github.com/llm-as-a-verifier/llm-as-a-verifier#self-verification-terminal-bench-21 and https://github.com/itigges22/ATLAS where you just have a less capable model attempt multiple shots for each step, and a critic pick one that looks best. Turns out this can dramatically improve capability.

    And what we're seeing as an overall trend right now is that a lot of work is moving in the direction of making coding harnesses more intelligent. The harness is basically a state machine which steers the model through a series of steps it has to complete. So, making the state machine more complex can go a long way in terms of steering. For example, the harness can check whether the model edited files, or ran tests. If it didn't then the harness can automatically inject a prompt nudging it to do that. And that's just a basic example. It's also possible to have the harness itself be adaptable. You can set up a model in a role of a supervisor, and then have it watch the workflow, then modify it on the fly. If it sees the implementer get stuck, it can try to intervene to correct the problem, or get it to do something else.

  • source
  • parent