I think the trick that's really underexplored right now is in leveraging the harness for the model. I'm playing around with an idea right now where I give a model a task and have it decide whether it thinks it can implement it in 30 lines or so, if not then it's asked to split the task up into subtasks, and so you end up building a tree of tasks that need to be completed. Once you get to the leaves you start implementing using TDD. If the model tries a few times and fails, then you ask it to split again. So, eventually it gets to the point where the task is small enough for it to do reliably. Once you build the leaves against a test, it's a small self contained piece of code that does a thing. Then you start bubbling up the tree where the node that did a split, now has to glue a few functions together, and work against a test.

My idea here is that we can aggressively control the context that the model has to think about. Any complex program can be broken down into smaller pieces with a function being the smallest unit of composition. So, if a model like Qwen can write around 100 lines of code reliably for most tasks, then that's really good enough to solve problems in general through decomposition.

Currently, people are trying to get LLMs to write code the way people do, and that's a mistake in my opinion. These things have different strength from a human developer, and we can make tooling around them that plays to those strength.