you are viewing a single comment's thread
view the rest of the comments
[–] 1 point 2 days ago (3 children)

I don't think a crossword puzzle generator requires all that much context. Maybe they will get there at some point. They are still consistently confidently wrong, which makes them feel super untrustworthy to me. If a coworker said to me "oh, you're right, this fixes it" 4 times in a row without fixing anything, I would not trust that coworker. LLMs do the same thing.

You can't even run them if you don't have enough system ram, but yeah, graphics vram is for sure very important.

  • source
  • parent
  • hideshow 3 child comments
  • [–] [S] 2 points 2 days ago* (2 children)

    LLM assisted development has nothing to do with trust at all. Think of it as a genetic algorithm. The model makes a rough approximation as a first pass, then it runs tests or specs, gets feedback, and iterates on it. And each iteration gets closer to what you specified. The better the specification that you provide is, the better the end result will be.

    Meanwhile, context isn't some abstract thing. It's the amount of tokens the inference engine can hold in memory. With the memory constraints you have, it's going to be tiny, which means the model isn't able to hold the whole solution in the context even for a small thing like a puzzle generator which will be thousands of lines of code.

    Good news is that pretty much any problem can be broken down into a series of steps. And then if you structure things well, each step can be treated as its own isolated problem which doesn't requite much context. With the example of a crossword puzzle, what you'd want to do would be to create an outline of the shape of the solution yourself, how the pieces fit together, and what the end result looks like, create tests for the functionality, maybe even files with stubs for the functions you want if you're working with a small model. Then you run an agentic harness like opencode where the model can iterate on the solution, run tests, and get feedback on what it's doing wrong. You create the scaffolding for the solution and you own the contract, while the model fills in the details for it.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 1 point 2 days ago (1 child)

    Trust is relevant in the same way it's relevant with old school blueprints, dependencies, and with other automation.

    In my experience the LLMs don't consistently get closer.

    A puzzle generator doesn't need to be thousands of lines of code by any stretch.

  • source
  • parent
  • hideshow 1 child comment
  • [–] [S] 2 points 2 days ago

    Not really because trust was never how we ensured software quality in the first place. What matters is the process, doing things like code reviews, having a good test harness, coming up with specifications, doing manual testing, and so on. LLMs don't really change anything here.

    I'd also love to hear how you actually use LLMs and what your workflow is, because they absolutely do get closer when you set up constraints correctly and run them in the agentic loop in the harness. I also don't know what you refer to by puzzle generator here, clearly you mean something simpler than I assumed. If you want to write out what the specific requirements are, I can show you what Qwen 3.8 comes up with on my machine for them.

  • source
  • parent